The #develop teamblog
#  Wednesday, September 26, 2007

[Warning: long post. Short version: Building in SharpDevelop goes multi-threaded, scroll down for benchmark results]

SharpDevelop contains a rewritten build system. It has some new features that might be interesting for AddIn developers, but one new feature is directly useful for users, or more specifically, users with multi-core CPUs.

Here a quick history on build systems in SharpDevelop:
SharpDevelop 2.0 (the first version to use MSBuild), compiled solutions by telling MSBuild to build the .sln file.
SharpDevelop 2.1 and 2.2 compile solutions by telling MSBuild to build the projects individually.

When writing the build engine for SharpDevelop 2.1, I tried to make it multi-threaded so that two projects that are not dependent on each other can be built in parallel. However, this did not work: MSBuild makes use Environment.CurrentDirectory; and if two MSBuild instances run in the same process at the same time, they overwrite the CurrentDirectory and cause the other instance to search files in the wrong location.

Now I solved this problem by moving the build into worker processes. However, this means that all MSBuild events are serialized and remoted to the host process, where SharpDevelop AddIns need to have access to them - e.g. the CodeAnalysis AddIn registers itself as MSBuild logger, waits for the TaskStarted, ErrorRaised, WarningRaised and TaskFinished events and fixes certain FxCop error messages (some FxCop errors are not associated with a line number, but with a class/member name). Other SharpDevelop AddIns might decide that certain MSBuild events should be displayed in some form, e.g. in the output window. The only way to give AddIns access to all MSBuild events like they had before is to serialize all events and send them to the SharpDevelop process. When doing a build where there's relatively few work but lots of events (e.g. "Clean solution"), the worker is sending up to 1 MB/s of serialized events to the host.

This had a noticeable impact on performance, so I've kept the first worker in the SharpDevelop process, and only additional workers get their own process. So if you use only one worker, you won't see any worker process and should get the same build performance as SharpDevelop 2.x.

So even though building a project has multiple parts, of which some are waiting for the disk and others are waiting for the CPU; the overhead of using a worker process will cause the build to get slower. (MSBuild has to check if input files changed, start the compiler; the compiler will load input files, actually compile, write the output assembly; and some other stuff happens in the obj\ directory that you probably don't want to hear about)

But if you have a dual core processor, the CPU overhead doesn't hurt: a little overhead for one core is better than not using that core at all (this is assuming the compiler you use is not multithreaded).

But enough talking, let's get some benchmark results:

I used SharpDevelop to compile the source code for SharpDevelop 2.2.1:
AMD 3500+ (single core), 2 GB RAM, Vista Ultimate (32 bit), Areo enabled:
   One worker: 34.3, 34.1, 34.6, 34.7, 34.3 : Average 34.4 seconds
   Two workers: 39.6, 38.7, 38.1, 37.0, 36.6 : Average 38.0 seconds

 Build (no changes, so it's not compiling, but just checking everything for changes):
   One worker:  8.5,  8.0,  8.3,  8.0,  8.0 : Average  8.2 seconds
   Two workers: 13.4, 13.9, 14.0, 12.8, 13.1 : Average 13.4 seconds

The number of MSBuild events (Target started, Task started, Log Message...) is not much different between Rebuild and Build (no changes), so we get a constant overhead of about 5 seconds for the SharpDevelop solution on my desktop computer with a single core processor.

Here finally is the interesting part: Test results from my dual core notebook:

Core Duo T2300, 1 GB RAM, Vista Home Premium (32 bit), Aero disabled:
   One worker: 35.2, 34.9, 35.2, 35.0, 34.3 : Average 35.1 seconds
   Two workers: 25.6, 23.9, 24.2, 23.5, 23.8 : Average 24.2 seconds

 Build (no changes):
   One worker: 7.4, 7.1, 7.2, 7.8, 7.1 : Average 7.3 seconds
   Two workers: 6.9, 6.8, 6.7, 6.6, 6.2 : Average 6.6 seconds

For build (no changes), the overhead plays a big role, but using two workers still made the build a little faster.

But for the real job when there's something to compile, using two workers really rocks: My build is now 30% faster.

SharpDevelop 3.0 allows you to set the number of workers under Tools > Options > Project and solution > Number of projects to build in parallel. The default value for single-core processors is 1, for other processors it's 2. I don't know if going above 2 will have benefits for quad-core users as the overhead (both CPU for sending events to the host and memory because of the additional processes) increases with the number of workers.

Please leave a comment here with your test results - I'm looking for feedback from other multi-core users.

And if you're doing your own benchmark, some notes on how I did my benchmark:
- make sure enough RAM is free so that not only the system does not swap, but Windows has enough room to keep the files you are compiling in memory (this is generally a good idea if you want your system to be fast)
- don't count the first test run as the files are not yet in memory there
- reuse the same SharpDevelop instance for all test runs
- if you leave less than 60 seconds between test runs, SharpDevelop will re-use the worker process, otherwise it is shut down and has to be restarted. I did my tests re-using a single worker process. It might be interesting to see if the overhead of restarting the worker process is worth keeping it longer than 60 seconds.

Categories: Daniel
Wednesday, September 26, 2007 9:05:40 PM (GMT Daylight Time, UTC+01:00)  #    Comments [0]


Comments are closed.

© Copyright 2014 SharpDevelop Core Team

Subscribe to this weblog's RSS feed with SharpReader, Radio Userland, NewsGator or any other aggregator listening on port 5335 by clicking this button.   RSS 2.0|Atom 1.0  Send mail to the author(s)


Copyright ©2000-2009 IC#Code. All rights reserved. Projects sponsored by AlphaSierraPapa.