SharpDevelop uses the MSBuild libraries
for compilation. But when you compile a project inside SharpDevelop, there's more
going on than a simple call to MSBuild.
SharpDevelop does not pass the whole solution to MSBuild, but performs one build for each project. This is done to give SharpDevelop more control - e.g. we can pass properties to MSBuild per-project. For example, when you click "Run code analysis on project X", we'll first build all dependencies of X normally, and then project X with code analysis enabled. A normal MSBuild call (e.g. if you use it on the command line) would perform code analysis on all dependencies of X, too.
When calling MSBuild on a project on the command line, MSBuild will find all referenced projects and build them recursively. We have to prevent this kind of recursive build inside SharpDevelop, as we already took care of the referenced projects. We don't want MSBuild to check referenced projects for changes repeatedly (this would dramatically slow down builds), and of course we need to prevent MSBuild from running stuff like code analysis on the dependencies. Fortunately, the Visual Studio team had the same requirement, so SharpDevelop can simply set the property "BuildingInsideVisualStudio" to true to disable recursive builds.
However, using that property, MSBuild will call the C# compiler on every build, even if no source files were changed. This is desired in Visual Studio - VS has its own "host compiler" that does change detection. But it's bad for SharpDevelop. As a workaround, we override the MSBuild target responsible for this and fix it. This is an in-memory modification to your project file; it never gets saved to disk.
Another point where we use this kind of in-memory modifications is for importing additional .targets files in your project. Some AddIns in SharpDevelop do this to add new features to the build process - for example the code analysis AddIn.
Now enter parallel builds. It would be nice to be able to call MSBuild on multiple threads and compile projects in parallel. Unfortunately, that's not possible. MSBuild uses the process's working directory as a global variable. In one process, only one build can run at a time. Even worse: if you have MSBuild in your process, all your other code must deal with concurrent changes to the working directory.
In .NET 3.5, Microsoft introduced the "/m" switch in MSBuild. This makes MSBuild create multiple worker processes (usually one per processor), enabling concurrent builds. Unfortunately, this feature is exposed in the MSBuild API only through a single method, and that only allows several project files from the hard disk in parallel. It does not support in-memory projects; it cannot even compile projects in parallel if there are dependencies. Microsoft solves the latter problem by separating the project in the solution into 'levels' which depend only on projects from the previous level. However, this doesn't mix well with the way building is integrated in SharpDevelop - we don't use levels but do a kind of topological sort on the dependency graph, and not all SharpDevelop project have to use MSBuild; AddIn authors could choose their own project format and build engine. In the end, I had to create my own build worker executable.
Nowin .NET 4.0, Microsoft created a completely new MSBuild API. The 'level' problem is solved: the new API allows adding new jobs to a running build. It also seems like it is possible to build in-memory projects in parallel now. But as it turned out, in-memory changes only work in the primary build worker (in-process), all other workers load the file from the hard disk and ignore our changes.
Instead of going back to our custom build worker, however; I decided to find a different solution for handling our modifications. Microsoft.Common.targets contains several extensions points adding custom .targets files by setting a property. A good solution for added a custom new target might be setting "CustomAfterMicrosoftCommonTargets" to the name of the .targets file. However, this might conflict with projects that already use this feature, so instead I chose "CodeAnalysisTargets". SharpDevelop comes with its own code analysis targets, so it doesn't hurt if we disable the Microsoft targets.
So in the end, the solution is trivial: create a temporary file containing only our modifications and set a property to tell MSBuild to pick up that file.
Why couldn't I simply write the project file including the modifications into a temporary file? The MSBuild reserved properties would point to the temporary file and custom build events using those properties would likely fail.
SharpDevelop does not pass the whole solution to MSBuild, but performs one build for each project. This is done to give SharpDevelop more control - e.g. we can pass properties to MSBuild per-project. For example, when you click "Run code analysis on project X", we'll first build all dependencies of X normally, and then project X with code analysis enabled. A normal MSBuild call (e.g. if you use it on the command line) would perform code analysis on all dependencies of X, too.
When calling MSBuild on a project on the command line, MSBuild will find all referenced projects and build them recursively. We have to prevent this kind of recursive build inside SharpDevelop, as we already took care of the referenced projects. We don't want MSBuild to check referenced projects for changes repeatedly (this would dramatically slow down builds), and of course we need to prevent MSBuild from running stuff like code analysis on the dependencies. Fortunately, the Visual Studio team had the same requirement, so SharpDevelop can simply set the property "BuildingInsideVisualStudio" to true to disable recursive builds.
However, using that property, MSBuild will call the C# compiler on every build, even if no source files were changed. This is desired in Visual Studio - VS has its own "host compiler" that does change detection. But it's bad for SharpDevelop. As a workaround, we override the MSBuild target responsible for this and fix it. This is an in-memory modification to your project file; it never gets saved to disk.
Another point where we use this kind of in-memory modifications is for importing additional .targets files in your project. Some AddIns in SharpDevelop do this to add new features to the build process - for example the code analysis AddIn.
Now enter parallel builds. It would be nice to be able to call MSBuild on multiple threads and compile projects in parallel. Unfortunately, that's not possible. MSBuild uses the process's working directory as a global variable. In one process, only one build can run at a time. Even worse: if you have MSBuild in your process, all your other code must deal with concurrent changes to the working directory.
In .NET 3.5, Microsoft introduced the "/m" switch in MSBuild. This makes MSBuild create multiple worker processes (usually one per processor), enabling concurrent builds. Unfortunately, this feature is exposed in the MSBuild API only through a single method, and that only allows several project files from the hard disk in parallel. It does not support in-memory projects; it cannot even compile projects in parallel if there are dependencies. Microsoft solves the latter problem by separating the project in the solution into 'levels' which depend only on projects from the previous level. However, this doesn't mix well with the way building is integrated in SharpDevelop - we don't use levels but do a kind of topological sort on the dependency graph, and not all SharpDevelop project have to use MSBuild; AddIn authors could choose their own project format and build engine. In the end, I had to create my own build worker executable.
Nowin .NET 4.0, Microsoft created a completely new MSBuild API. The 'level' problem is solved: the new API allows adding new jobs to a running build. It also seems like it is possible to build in-memory projects in parallel now. But as it turned out, in-memory changes only work in the primary build worker (in-process), all other workers load the file from the hard disk and ignore our changes.
Instead of going back to our custom build worker, however; I decided to find a different solution for handling our modifications. Microsoft.Common.targets contains several extensions points adding custom .targets files by setting a property. A good solution for added a custom new target might be setting "CustomAfterMicrosoftCommonTargets" to the name of the .targets file. However, this might conflict with projects that already use this feature, so instead I chose "CodeAnalysisTargets". SharpDevelop comes with its own code analysis targets, so it doesn't hurt if we disable the Microsoft targets.
So in the end, the solution is trivial: create a temporary file containing only our modifications and set a property to tell MSBuild to pick up that file.
Why couldn't I simply write the project file including the modifications into a temporary file? The MSBuild reserved properties would point to the temporary file and custom build events using those properties would likely fail.
