I recently sat in on a discussion that Jeffrey Richter, a co-founder of (http://Wintellect.com and contracting...
vendor for Microsoft's CLR and Version Architecture team, gave about the future of assembly versioning for the next version of the Windows operating system, codename Longhorn. This was the first time this information was being discussed outside Microsoft. Here is a summary of that discussion.
Current State of DLL Versioning
Currently, Windows uses just the filename of a DLL to determine the library to load even though version information can be included inside the file. If, at a later date, a library is installed with the same name, in the same directory, the file is overwritten. This can cause applications to stop working correctly once another application is installed. This problem has affectionately been labeled "DLL Hell."
The .NET Framework was developed with the idea to fix this problem by taking a 180 degree turn in how versioning is handled. With .NET, the libraries, called assemblies, are identified by a strong name, which includes the filename, version, culture and public key. This allows all assemblies to be exactly identified since assemblies from different companies, even with the same filename, version and culture, will always have different public keys. Newer assemblies can then be installed side-by-side with older versions, without the risk of overriding each other.
The CLR loads the specific assembly that an application was built with. If changes are made to an assembly, like bug or security fixes, the application will need to be recompiled to use the newer version. To solve this problem, Microsoft provided the ability to create policy files that allowed an application to use a newer DLL version without the need to recompile. However, policy files are complicated to create and many developers have shied away from using them. This problem has affectionately been labeled "Policy Hell" or "Versioning Hell".
The ultimate goal for Microsoft is to allow developers to modify and ship new DLL versions while guaranteeing 100% backward compatibility. "If you think about this problem for a while and try to come up with possible solutions," Richter said, "you will soon realize that the problem cannot be solved perfectly. It is impossible to allow just 1 bit in a file to change while maintaining 100% backward compatibility." For this reason, the DLL versioning plans for Longhorn are based on a compromise between the Windows and .NET ways of doing things.
What the Customer Wants
Microsoft has conducted studies to determine what their customers want when it comes to versioning. Their results have shown that the number one need of a customer is to be able to install one application and have it never break another application. Another result of the survey was that when installing an operating system, it should very rarely break an existing application. This surprised many at Microsoft that customers are willing to accept the fact that operating systems may not be 100% backwards compatible with existing applications. However, the customers also stated that when their application does break, they will need the ability to have the application use "interim roll back" and run. The term "interim roll back" will be discussed later in this paper.
Vision of Microsoft's Version Architecture Team
Given the fact that Utopia is unattainable and balancing that with what the customers want, the Version Architecture team set out to find the best solution. The first vision for the team was to develop a model that allows developers to devote more time to innovation and less to backwards compatibility as compared with previous versioning models. This means that applications can have more features and ship faster. Another vision is to make deployment of a new version of an application or component as simple as drag and drop. In other words, drop the whole notion of the overly complicated policy files. The last vision of the versioning model is when a new assembly breaks an existing application; the administrator can make the application work by temporarily using the old assembly.
When asked if these visions were mutually exclusive, and what the rationale behind these visions were, Richter responded, "Versioning is all about innovating (changing code) while maintaining backward compatibility (not changing code). As you can see, it's impossible to change code while not changing it and this is what makes versioning an impossible problem to truly solve. People will not pay for code that's not changing; they will pay for innovation. So innovation is important for Microsoft to sell new copies of Windows and, in fact, innovation is required for our entire industry to thrive. For this reason, I see versioning as one of our industry's biggest problems to overcome. Our visions come from significant thought and discussion with customers and are all about selecting the best set of defaults behaviors for the system to exhibit and what policy changes to offer and how these policy changes get exposed."
Versioning at a High Level
Microsoft broke the problems into three areas and determined how to solve each one.
Problem 1: Componentization
Developers today are at a crossroads when it comes to how to break their code into different assemblies. On the one hand, if they break their code into many different assemblies to reduce file size, the complexity of management of the versions of the DLLs will grow, causing the application to become more fragile. On the other hand, creating one monolithic assembly for the entire application creates a large file to manage. Every time an update is needed, the large file must be sent over the wire.
Microsoft's solution to this problem is to provide dependency analysis. The dependency analysis is a way to know what needs what at design, build, and load time. The model would understand all the assemblies needed for an entire application and not just limited to the needs of a single assembly.
Problem 2: Innovation versus compatibility
The versioning model needs to reduce the cost of maintaining backwards compatibility while being able to ship new components. Developers are continually questioning what impact on shipping a new feature of an application will have with breaking existing code. This stifles their willingness to innovate new features.
To solve this problem, Microsoft wants to break the assemblies into two types; Platform and Library. The Platform assemblies will contain components that must stay backwards compatible. Changes to these assemblies will be limited to adding components and never removing any. All other components will be incorporated into Library assemblies. By definition, these assemblies will allow innovation, with the understanding that applications may need revisions to make use of the newer versions. This provides the freedom to develop enhancements without the need to guarantee backwards compatibility.
Problem 3: Servicing and Management
Currently, when assemblies are installed on a machine, there is no guarantee that another application could stop working because a file was overwritten with the newly installed assembly. The solution to this problem is to provide a versioning/servicing number to guarantee that one file does not overwrite another.
Versioning Model: Platform and Library Assemblies
The new versioning model will allow developers to choose the type of assembly they build by assigning an attribute to an assembly (see Figure 1). The current prototype uses the attribute name AssemblyFlagsAttribute, assigning it a value from one of the AssemblyNameFlags enumerated type. Developers will have an understanding of the versioning model of the assemblies that they use. Platform assemblies will have an underlying contract to never break backwards compatibility whereas Library assemblies will not have this same guarantee. The development tools will have an understanding of the versioning model and warn the developer if they violate the rules
According to Richter, "A developer using Platform types expect their code to work with newer assembly versions. Developers of Platform assemblies should do significant testing to ensure that new version do not break existing apps. Unfortunately, if a new version does break an app, this will only be detected by observing runtime behavior. If a Library developer decides to break behavior, the assembly should be assigned a different filename and there will probably be significant changes to the types in the assembly causing compilation errors. Users of a new version of a Library assembly will know immediately that breaking changes are likely because they have to change build settings (assembly references) and fix potential compiler errors."
Platform assemblies will allow component developers to define never changing object models and behaviors. This assembly type has been designed for interfaces, abstract base classes, and simple types. When a new version of an assembly is installed, existing consuming code of the assembly automatically uses the new version and continues to work. In addition, the consuming source code will not require modification in order to recompile. In order to achieve this, the developer must adhere to the following contract for platform assemblies:
Developers building platform assembly types must forever maintain backward compatibility: Library assemblies allow component developers to define self-contained object models and behaviors which may change significantly with new versions. Examples of these types of assemblies could be DirectX and ADO.NET. The object models for these classes are constantly in flux so the versioning model needs to be adaptable. When a new version of a Library assembly is installed, existing consuming code still uses the old version of the library and continues to work.
In order to use the new version of the library, consuming code must be rebuilt and may require source code changes. In order to achieve this, the developer must adhere to the following contract for Library assemblies:
Consuming code cannot pass an instance of an old library type where a new version is expected and vice versa: Libraries are considered the holy grail of versioning but there has to be some minimum base level of functionality that all components can assume. This is where the Platform assemblies fit in. They provide this minimum base level functionality. Since Platform assemblies require backwards compatibility and Library assemblies do not, then, by definition, when developing Platform assemblies, they can not have Library types in their APIs.
The versioning model can be related to the COM model. Platform assemblies are for the interchange purposes and to specify contracts. Library assemblies are implementations of Platform types. Think of this phrase: "Platform assemblies are to Interfaces as Library assemblies are to Implementation."
Starting in Longhorn, only one version of the CLR will be allowed on the client's machine. This will eliminate the current problem where a host application loads a component and if this first component gets to select the CLR version, then non-deterministic behavior and compatibility problems ensue. When accused of breaking the promise of side-by-side compatibility with prior versions of the runtime, Richter responded, "Not really; the CLR will be single but the frameworks are still side-by-side. We have always kept the GC, JIT compiler, Code Access Security system, assembly loader, etc. backwards compatible from v1.0 to v2.0. Longhorn will still ship v1.0, v1.1, and v2.0 of the Fx [also known as the Base Class Library] libraries."
Platform assemblies will automatically roll forward. This will allow all applications and components to use the latest version installed. If an application or component fails, an administrator can apply interim roll back for that application or component. This interim roll back is not a permanent policy about what assemblies should be loaded for the application. When the next release of the assembly is installed, the interim roll back policy will automatically be canceled and the application will begin using the newest version. The reason for this design is based on two assumptions made by Microsoft. The first assumption is that new releases, statistically, don't break applications so, the odds are good that a new version of an assembly will not break an application. The other assumption is that if an application does break, probabilistically, the application will work both under the interim roll back and when the next future release becomes available.
Platform assemblies are further broken down into three types. The least agile type is Machine Platform. There can only be one version of the classes contained in the assemblies per machine and this assembly can not be rolled back. Some examples of Machine Platform classes are System.Object, System.AppDomain, and the CLR itself. The next Platform assembly type is Process Platform, which allows one class version per process. An example of a class contained in this type of assembly is the System.Web.UI.Page class. The third Platform assembly type is the AppDomain Platform, which as the name implies, allows one class version per AppDomain.
The System.Windows.Forms.Control class is an example of a class contained in this assembly type. Both the Process Platform and AppDomain Platform assemblies will have the capabilities to provide an interim roll back. There can be many Library assembly versions in the same AppDomain. Some examples of classes contained in a Library assembly would be System.Windows.Forms.Control-derived and System.Web.UI.Page-derived types.
To allow developers to take advantage of this new versioning model when released with Longhorn/Orcas, Microsoft is encouraging developers to adhere to the following rules when writing code today.
The first rule is to not use less compatible types in your DLL's public API; doing this can lead to a MissingMethodException. Some examples of this rule are a class defined in a Library assembly can expose Platform classes. However, a class defined in any Platform assembly cannot expose Library classes. Along the same lines, a class defined in a Process Platform cannot expose AppDomain Platform or Library classes. In the Longhorn/Orcas timeframe, the compilers will be enhanced to report these types of violations.
The following principles help determine which assembly type to put a class into:
|Integration||Is the class deliberately designed to be passed between independently versioned add-ins?||Platform assembly||System.DateTime|
|Incompatible Evolution||Is there reason to believe that the class will evolve incompatibly over time or that the cost of backward compatibility might be high over time?||Library assembly||DataGrid|
|Consistency||Each major tree of classes starts with a Platform abstract base class and Platform interfaces. As additional functionality is added by subclassing there should be a clear line where types move from Platform type to Library type.||Platform assembly and Library assembly||System.Web.UI.Control class is Platform type, but specific value-added controls are Library type.|
Classes should be Library assembly types unless there is a strong argument to the contrary. The long-term maintenance cost for Platform assembly types is far higher than for Library types. Platform types are what make upgrades risky and block adoption. Classes that are not passed around should be Library types, like application code. Most of the WinFx types are expected to be passed around and will be targeted for the AppDomain Platform. Interfaces and abstract classes with little implementation are good candidates for Platform types. Concrete implementations of interfaces are good candidates for Library types. Finally, sub-classes of Platform types that provide considerable additional functionality for ease-of-use are good candidates for Library types.
The current naming convention for this model is the following:
|Library||Version number is in the filename||System.Web.Mail.v3.0.dll|
|AppDomain Platform||no decoration in the filename||System.Web.dll|
|Process Platform||PW is in the filename||System.Web.Hosting.PW.dll|
|Machine Platform||MW is in the filename||System.MW.dll|
For Library assemblies, the developer can break current behavior. The CLR will always load the version that the application was compiled against. The application must opt in to the newer version. As an author of this type of assembly, the behavior changes need to be documented and communicated to the users. From a business perspective, avoid significant breaks in behavior so as to not cause significant source code changes. This could cause resentment every time a new version of the library is released.
The CLR will always bind to the same major/minor version as what it was compiled against. For example, if the version number is 220.127.116.11, the major/minor version is 1.2. However, the versioning model will support servicing. This allows the developers to build a new version of the assembly and only change the build/revision portion of the assembly. In the previous example, that would be 3.4. The CLR will always load the latest "servicing" version of an assembly, allowing developers to provide updates to assemblies without requiring publisher policy files or binding redirects.
Platform assemblies do not break the object model so developers can add members but not remove members. If somehow code gets into the latest version that causes a breakage in the application, administrators can perform an interim roll back until the next release, where a roll forward will occur. When the version number is changed, the latest version will always be loaded by the CLR. Major/minor version changes will allow interim roll backs and should be used for feature enhancements. Any changes made to the build/revision portion of the version, will require and install/uninstall, not an interim roll back.
Clearly, getting the assembly versioning story "right" is a Herculean task, and not one easily accomplished. Developers remember all too well the "solutions" proposed in the early Windows timeframes, and the "DLL Hell" that resulted. The 1.0/1.1 versions of the .NET Framework took a healthy step in the right direction, but relied too strongly on policy, creating the subsequently-named "Policy Hell" that face .NET administrators and developers alike. Microsoft now seeks to address these problems further, by leveraging what Windows gives us today, but readers are cautioned that, as with any discussion of technologies in "beta" status, everything said here is subject to change in subsequent releases.
Microsoft teams are striving to find the "right" balance between giving developers the ability to run their programs on new platforms, assured of backwards compatibility, and the agility to adapt to the changing needs of users, which might incur breaking changes. Finding this "sweet spot" will take time, dedication, and a lot of feedback from .NET developers all over the world. For now, developers just need to continue to write assemblies that maximize component concepts, keep an eye on the proposals coming from Redmond, and offer thoughts and feedback as much as possible.
About the authors
Cathi Gero is the co-founder and development director of Prenia Corporation, providing custom software applications to businesses since 1987. Her expertise lies in developing web-based and desktop .NET applications, project mentoring, custom on-site .NET training, and development of database solutions using Visual Studio .NET, SQL Server, Visual FoxPro and Crystal Reports.
Jeffrey Richter is a co-founder of Wintellect, a training, debugging, and consulting firm dedicated to helping companies build better software, faster.