Recently I’ve been working with some systems that don’t build as fast as I’d like. I’ve therefore taken a project and made some changes to attempt to optimise the build speed. This post discusses these optimisations.

In gathering these numbers I’ve run each build multiple times and discarded the first of each builds to account for setup times and the like. I’ve normalised each figure against the time taken by the initial unmodified build runs. This isn’t particularly scientific but it’s adequate for this usage. The build run includes the time to build the system and to run all the unit tests. This is accomplished using a pre-existing MSBuild script build for this purpose.

From the baseline the first modification I made was to consolidate a number of the projects. This system had a number of relatively small (single class) projects. So the first step was collapsing this code into another project. This was about an 11% reduction in the number of projects. From this I get a normalised time of 0.979336. This is not a particularly big improvement but it’s a start.

At this point I’ll diverge slightly and justify collapsing the projects. There is an argument that keeping these things separate is justified for reasons of separation of concerns. I feel in this case that the argument is incorrectly conflating logical and deployment separation. As the system is used there is no need to deploy these elements separately. Having a separate project is therefore a cost that is not necessary to bear. It is still possible and recommended to separate the classes logically using namespaces. In this case this provides the appropriate separation of concerns without involving unnecessary deployment concerns.

The next optimisation step was to alter where the build process places build outputs and how dependencies are copied. By default Visual Studio has each project build into its own bin directory, copying dependencies (including other projects in the same system) as required. I have previously seen discussions suggesting that altering this behaviour to copy all projects into a single directory could have performance benefits. I have therefore made the following modifications to the system:

  • All non-test projects build into a common buildresult directory
  • Copying adjusted for non-test projects so that only the first project in the build order will copy a dependency. No projects in the solution are copied as dependencies into this directory as they will be placed into it by default.
  • All test projects build into a common testresult directory
  • Copying adjusted for test projects so that only the first project in the build order will copy a dependency. Only non-test projects are copied into the testresult directory.

This adjustment has resulted in a normalised build time of 0.76553 on average. This is a significant improvement over the initial build time. As a bonus the system is now in a common directory from which it is significantly easier to produce a packaged format.

As a final step I’ve switched the unit test framework from MSTest to MbUnit (version 3 using Gallio). This was relatively trivial and involved:

  • Removing the MSTest references and substituting the MBUnit and Gallio assemblies
  • A search and replace to change the attributes. These mapped one-to-one so no tweaking was required.
  • Fixing a couple of instances where the arguments to AssertIsInstanceOfType were reversed (MSTest has inconsistent ordering here)
  • Tweaking the cleanup of a few tests that were breaking
  • Changing the build script to use the Gallio MSBuild task to invoke the tests.

The normalised time taken to build after this switch was 0.44929. This means that in a relatively short amount of time I’ve been able to reduce the build time of this system by over 50%. And I’ve got a better test framework as well, which is nice. There are however some caveats:

  • In general development the changes I’ve made to how the dependencies are copied may be problematic and require some changes.
  • Management of dependency copying will be needed whenever dependencies are modified or whenever a project is added or removed. This should be relatively infrequent but it does exist as a cost.
  • The boost from the test framework switch is likely in part due to an improvement in how they are invoked. The initial build script invokes MSTest.exe on each individual test project which involves a certain amount of overhead. Gallio provides a test runner that runs all the tests in a single process, resulting in much lower overhead. I still consider this a win for Gallio as the options for interacting with MSTest are limited but you should consider whether the mechanism you use to run tests is more efficient.

In summary:

  • Small improvements are possible if you can consolidate assemblies, treating them as units of deployment not logical separation.
  • Significant improvements are possible by building projects into a common directory and managing how dependencies are copied.
  • Significant improvements may also be possible by switching test frameworks, depending on your initial configuration.