The #develop teamblog
 Tuesday, April 01, 2008

Starting with version 3.0.0.3010, the C# code completion in SharpDevelop has support for implicitly typed lambda expressions.

Given a variable "IEnumerable<MyClass> items" and the "Select" extension method from LINQ, typing "items.Select(i => i." now shows the members of MyClass. And if the result of the Select call is assigned to an implicitly typed variable, SharpDevelop is now able to infer that the variable has the type IEnumerable<return type of the lambda expression>.

Unlike all other expressions in C#, the type of a lambda expression cannot be inferred just from by at the expression itself (and the variables used by the expression). To resolve lambda type parameters, we also need to look at the context where the lambda is used. Currently, not all contexts are supported by code-completion, you can find the list of known problems in our bugtracker (Component: DOM / Resolver). Should you find anything where code-completion does not work correctly which is not in that list, please file a bug report in our forum.

The most commonly used context for lambda expressions is method calls, and this is also the most difficult thing to support. It's easy when the method has a clear signature like "void M(Func<int, string> f)", since then SharpDevelop can infer the lambda parameter types directly from the delegate type. But most of the time, things aren't that easy. For example, the signature of the Select method is "IEnumerable<R> Select<T, R>(this IEnumerable<T> input, Func<T, R> f)". Here, SharpDevelop needs to first infer what T is, then it can know what the lambda parameter types are, and only after that it can resolve the lambda expression to infer what R is.

But when the method has multiple overloads, things can get even more messy:
When a method has to overloads "void M(Func<string, int> f)" and "void M(Func<int, int> f)", it is valid to call them like this: "F(i=>i.Length)", "F(i=>i+i)". In the first call, i is a string; in the second, it is int. What SharpDevelop needs to do here is to infer the lambda parameter types for each overload separately, infer the lambda's return type; and then check that against the delegate's signature to see which overload was the correct one.

i=>i.Length is a resolve error if i would be int, but returns the expected int if i is string; so i must resolve to string.
i=>i+i returns a string if i would be string, but returns the expected int if i is int; so i must resolve to int.

Note that because there's no way to tell the type of i before the lambda expression is completed, you cannot expect that SharpDevelop gives you correct code completion for it. "Length" will not be included in the code-completion list for i when you type ".", because at that point, the method call is "F(i=>i)", and i is thus an int. But after the expression is written, SharpDevelop will show a tooltip for "Length", and features like "Go to definiton" and "Find references" will work.

Categories: Daniel
Tuesday, April 01, 2008 8:13:48 AM (W. Europe Standard Time, UTC+01:00)  #    Comments [0]

 



 Sunday, January 27, 2008

In SharpDevelop 1.1, the IClass interface had a property that was used in several places in the code:
Once added to a project content, it was immutable. This was not enforced, not even documented. It just happened that no one changed IClass objects except for the code constructing them. After being added to a project content, a class could be removed or replaced by a new version, but if some code still held a reference to an old instance, it could safely access the members without worrying that an update to the class on another thread changed something.

IClass objects are accessed without locks all over the place on the main thread during code completion, and updates on the parser thread should not interfere with that.

However, because I didn't understand this, I broke it in SharpDevelop 2.0. My implementation of partial classes works as following: each file (compilation unit) contributes a part of the class as IClass object representing that part of the class.
Once multiple files register a part of the class in the project content, the project content creates a compound class. This compound class combines the members of all the parts. When updating a part, only the IClass for that part was recreated, and the existing CompoundClass was updated.

The CompoundClass, which could be in use by multiple threads, changed values. To quote Wes Dyer: "Mutation often seems to just cause problems."

Now, that happened to work correctly for quite some time. Most code iterating through a class' members did this with
foreach (IMethod method in c.Methods) {
}

where c is an IClass (possibly a CompoundClass). An update of the compound class happened to recreate the List<IMethod> behind the c.Methods property, so this code continued to work as expected.

However, in the quick class browser (the two combo boxes above the code window), there was code similar to this:
list.AddRange(c.Methods);
list.Sort();
list.AddRange(c.Properties);
list.Sort(c.Methods.Count, c.Properties.Count); // sort the properties without mixing them up with the methods

Suddenly, due to the addition of partial classes, this became a race condition waiting to happen. But it was found in time for the SharpDevelop 2.0 release, and I fixed the crash.
But I didn't know much about immutability back then, so what I did was the worst fix possible:
lock (c) { ... }
And in CompoundClass, during update of the parts: lock (this) { ... }

Now, this is not only bad because it's fixing the symptom instead of the problem and it leaves the possibility for similar problems elsewhere in the code - though it might have been the only instance of the problem, since no other crashes due to this have been found while SharpDevelop was using the fix (all 2.x releases use it, including the current stable release, SharpDevelop 2.2.1).

But multi-threading (without immutability) is not hard, it's really, really hard. So it's not really surprising that some day, I found this code to deadlock.

So where's the deadlock?
First I must tell you the other lock we're colliding with: every project content has a lock that it uses to ensure that GetClass() calls are not happening concurrently when the list of classes is updated. So the parser thread acquires the project content lock and then the CompoundClass lock to update the CompoundClass.

But why would the AddRange / Sort code deadlock with this? The comparer used for list.Sort() sorts the members alphabetically using their language-specific conversion to string. In the case of methods, this includes the parameter list, including the parameter types.

What you need to know here is that type references (IReturnType objects) are not immutable - they need to always reference the newest version of the class, as we cannot afford rebuilding all IReturnType objects from all classes in the solution whenever any class changes. Now remember that C# allows code like "using Alias = System.String; class Test { void Method(Alias parameter) {} }".

In this case, the quick class browser correctly resolves the alias and reports "Method(string parameter)". This means that our Sort() call actually sometimes needs to resolve types! And resolving types works using IProjectContent.SearchType, which locks on the project contents' class list lock. And that's our deadlock.

I think it's near impossible to find this kind of deadlock until it occurs and you can see the call stacks of the two blocked threads.
Remember that the actual Method->string conversion and the type resolving is language-specific; it may or may not happen to take a lock for other language binding AddIns.

I fixed the deadlock on trunk (SharpDevelop 3.0) a few months ago by removing the lock on CompoundClass and instead doing this:
list.AddRange(c.Methods);
list.Sort();
int count = list.Count;
list.AddRange(c.Properties);
list.Sort(count, list.Count - count);

It's still a hack, but this doesn't have any side effects (like taking a lock). And it works correctly under our (new) rule (undocumented rule, aka. assumption) that multiple c.get_Methods calls may return different collections, but the collection's contents don't change.

"foreach (IMethod method in c.Methods) { ... }" is safe, but "for (int i = 0; i < c.Methods.Count; i++) { IMethod method = c.Methods[i]; ... }" can crash.

But that's quite a difficult rule compared to "IClass never changes". So after reading Erip Lippert's series on immutability, I finally decided to make IClass immutable again.

It's "popsicle immutability", that means IClass instances are mutable, but when the Freeze() method is called, they become immutable. And this time, immutability is enforced, trying to change a property of a frozen IClass will cause an exception. Adding an IClass to a project content will cause it to freeze if it isn’t already frozen, so it's guaranteed that IClass objects returned by GetClass or by some type reference are immutable.

Categories: Daniel
Sunday, January 27, 2008 12:30:59 AM (W. Europe Standard Time, UTC+01:00)  #    Comments [0]

 



 Monday, November 19, 2007

Microsoft just released Visual Studio 2008 and the .NET Framework 3.5.

Download .NET Framework 3.5

To develop applications for the .NET Framework 3.5, download SharpDevelop 3.0 from the build server.

Note that SharpDevelop 3.0 are not release quality builds, especially the integrated debugger is very unstable. As currently no one is working on the debugger (hint: you can help!), I disabled it in SharpDevelop 3.0.0.2745 so that people trying SharpDevelop 3.0 do not immediately get exceptions when they try run their programs. To re-enable the debugger, open C:\Program Files\SharpDevelop\3.0\AddIns\AddIns\Misc\Debugger\Debugger.AddIn.addin in a text editor and remove the <DisableAddIn .../> node.

Categories: Daniel
Monday, November 19, 2007 10:35:48 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [0]

 



 Monday, October 01, 2007

SD2-1234, titled "Create common way to handle in-memory representations of files that have multiple views" is the issue tracker entry behind a major refactoring of the IViewContent interface.

The major new feature introduced is that now it is possible to open a file in multiple view contents at the same file (using "Open with"). Both IViewContent instances will edit the same underlying file - when you switch between them, one view content will display the unsaved changes of the other; pressing Ctrl+S in one of them will save both.

Now how is this possible? There must be some data structure shared by both view contents. Since the view contents might not know anything about each other (both could be independently developed AddIns), this data structure must work on the lowest possible level: bytes. It's simply a MemoryStream.

Of course the view contents cannot serialize their content on every change; so a view content may have local changes in a higher-level data structure (text editor buffer, list of entries in resource file, etc). To ensure that such changes are correctly synchronized between view contents showing the same file, I introduced the concept of an active view content. The active view content is the only view content that may have such local changes. Thus, the active view content always has the most recent version of the file, and it is the only view content that is guaranteed to have the most recent version.

To represent files, I created the OpenedFile class. This class knows which view contents have opened it and knows which of them is the active view content. Usually every OpenedFile can have a different active view content; though it is possible for multiple OpenedFiles to have the same active view content if that view content opens multiple files - the refactoring also added support for view contents that are editing multiple files at the same time.

Now a quick overview of the SharpDevelop UI: The SharpDevelop main window is composed of a main menu, a tool bar, and the DockPanel of Weifen Luo's DockPanel Suite. In the dockable area, there are pads, which can be docked in groups at the sides of the SharpDevelop window; or can float somewhere (e.g. on a second monitor).
In the remaining space (not occupied by pads), workbench windows are displayed. The active workbench window can be chosen using the tabs at the top. There can be multiple visible workbench windows if a user drags a tab and docks it to the side of the remaining space.
Finally, each workbench window contains at least one view content. The active view content can be chosen using the tabs at the bottom.

So if you create a new Windows Application and open Program.cs and MainForm.cs, there are two workbench windows titled "Program.cs" and "MainForm.cs", but three view contents - "Program.cs - Source", "MainForm.cs - Source" and "MainForm.cs - Design".

Note that in the new IViewContent architecture, there are no secondary view contents. Now all view contents are equal, and they shouldn't care if they share a workbench window or not. Whether they will share a workbench window or not depends on how the view contents were created - there is still the difference between primary and secondary display bindings, which are responsible for creating view contents.

Now how do changes get from one view content to the other? It's quite simple: Whenever the user activates a view content (by clicking a tab at the top, by clicking a tab at the bottom, or by setting focus into another workbench window after the user docked a window to the side), that view content becomes the active view content for all files it has opened. The old active view content will be asked to save its content to a MemoryStream and the new active view content will be asked to load from that MemoryStream. This way, unsaved changes are transferred from one view content to another.

When the active view content for a file is closed, but there are other open view contents for that file, SharpDevelop will not ask the user to save the file. Instead, it will save the data from the view content being closed into a MemoryStream. After that, the OpenedFile has no active content. Only when one of the other view contents that have opened that file get activated by the user, that view content will load from the MemoryStream and thus will preserve unsaved changes from the closed view content. However, if the user closes all other view contents for that file without making them active (by middle-clicking, or using Window>Close all), SharpDevelop will ask the user if the file should be saved and write the MemoryStream content to disk if required.

The system sounds simple for view contents: they just have to be able to load and save; and it'll just work.

But it isn't that easy. The view contents must be able to load and save reliably at any time.

The user just did something invalid which cannot be saved and then switches to another view of the file? The view content is forced to save. It's not possible say "I don't want to save".

The user loads a .resx file in the text editor, changes something by hand that renders the file invalid XML, opens it in the resource editor, gets an error message, switches back to the text editor. Here the file is saved by the text editor, loaded in the resource editor, the user gets the error and switches back, the resource editor must save and the text editor will load again. The resource editor view content must support loading and saving invalid files unless you want this kind of round-trip to result in loss of user data.

If your view content is editing multiple files, it gets even more complicated: you must support loading and saving individual files reliably at any time, in any order. Sounds fun, right?

Well, if you don't get this right, the user looses data only when editing a file in multiple views simultaneously. In SharpDevelop 2.x, view contents were simply overwriting each other's data; in SharpDevelop 3.0 there's at least a chance that it works if all view contents are implemented correctly.

However, be warned that I didn't have the time to update all view contents in SharpDevelop to make use of the new model. There's still a class AbstractSecondaryViewContent that implements Load and Save so that they run through an underlying "primary" view content, so existing secondary view contents do not have to be completely rewritten (although they still need several changes). The text and resource editors are fine; use them to see how it should work. The forms designer does not yet use the new model, it uses AbstractSecondaryViewContent and still touches the Designer.cs file directly, resulting in bugs like SD2-1175.

But if you are writing a new view content, try to design it so that you can support loading and saving at any time. The AbstractViewContentHandlingLoadErrors class (which both the resource editor and the WPF designer use) can help you handling invalid files.

If your view content edits multiple files, it can get tricky to support loading and saving those independently. But if it is likely that a user will want to edit one that files separately while also using your multi-file view content, you will have to do it. It is possible in SharpDevelop 3.0, so that's an improvement over SharpDevelop 2.x.

By the way: the reason for all this is the settings designer (still not implemented): it edits both a .settings XML file and app.config, and it's very likely that the user has opened the app.config at the same time.

 

Post by Daniel Grunwald (we use the category to mark the author on this blog, but I'll repeat it from now on at the bottom of the post because some feed readers like Google Reader don't show the category)

Categories: Daniel
Monday, October 01, 2007 8:41:07 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [0]

 



 Sunday, September 30, 2007

Visual Studio 2008 uses MSBuild 3.5 which supports multi-targeting: you can use it to compile applications for .NET 2.0, .NET 3.0 and .NET 3.5.

But what happens if you open an MSBuild 2.0 project (Visual Studio 2005 or SharpDevelop 2.x project) in Visual Studio 2008?
Answer: The “Visual Studio Conversion Wizard” will pop up and “convert” the project to MSBuild 3.5. Though it does generate a fancy upgrade report that looks like source files have been converted, all it does for a normal Windows Application is the following:

  • In the .sln file, the solution version is set from 9.00 to 10.00, and the “# Visual Studio 2005” line is replaced with “# Visual Studio 2008”. This causes the Visual Studio loader to run VS 2008 instead of VS 2005 when the .sln is double-clicked.
  • In the .csproj file, the attribute ToolsVersion=”3.5” is inserted. This causes MSBuild to use the C# 3.0 compiler.
  • The property “<OldToolsVersion>2.0</OldToolsVersion>” is inserted, and two properties FileUpgradeFlags and UpgradeBackupLocation are introduced, but not set to a value. This has no effect on compilation.
  • In the .csproj file, the Resources.Designer.cs file got marked as “<DesignTime>True</DesignTime>”, which has no effect on compilation.
  • All files generated by a custom tool (Resources.Designer.cs and Settings.Designer.cs) get regenerated.
  • All other source files remain unchanged.

Note that even though the C# 3.0 compiler is used, the project is still compiled for .NET 2.0. The new C# 3.0 languages features are available, but LINQ cannot be used because it requires referencing System.Core.dll, which comes with the .NET Framework 3.5.
So the converter did only minor changes, but they prevent working with both VS 2005 and VS 2008 on the same project.

Currently, SharpDevelop does not use the multi-targeting model of VS 2008, but continues to use the model of SharpDevelop 2.x: The “Target Framework” setting in the project options allows you to choose the compiler to work with. If you choose C# 2.0 / .NET 2.0, SharpDevelop will not use ToolsVersion=”3.5” on the project, and will create a solution file version 9. Only if you choose the C# 3.0 compiler, it will make the project an MSBuild 3.5 project and mark the solution as version 10.

However, this introduces several problems:
C# 2.0 does not support embedding manifest files in assemblies, but C# 3.5 embeds a default manifest file automatically. Simply changing the target framework changes the manifest behavior.

MSBuild 2.0 does not check whether the assemblies you referenced are available in the target framework you have chosen. SharpDevelop can fix this by doing the check on its own.

The target framework is an option that normally can be dependent on the chosen configuration/platform. However the ToolsVersion attribute in MSBuild cannot be dependent on configuration/platform. If one configuration uses C# 3.0, all configurations will; even if they are explicitly set to C# 2.0.

WPF Applications created in SharpDevelop 2.2 use a different way to support XAML and resource compilation than MSBuild 3.5 uses. Such applications can be edited in SharpDevelop 3.0 and will continue to work even though they are set to C# 2.0 / .NET 2.0 in the options.  Setting these projects to C# 3.0 causes problems that can only be fixed by manually editing the .csproj file.

So what should we do?

  1. Keep it as it is: works for usual projects, has the problems mentioned above
  2. Do it like VS 2008, drop support for the C# 2.0 compiler and force conversion of the project
  3. Disable the “Target framework” combo box until the user runs a conversion tool for that project (which could be a button next to the target framework combo box)

1. “Keep it broken” has the advantage that it’s zero work for us
2. Would be the easy to implement, but means that you cannot use SharpDevelop 3.0 if your co-worker still uses SharpDevelop 2.x or Visual Studio 2005
3. Would be more work than 2, but is the most flexible solution

So do you think the ability to edit VS 2005 projects is worth the effort of implementing 3 instead of 2?

Categories: Daniel
Sunday, September 30, 2007 5:26:53 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [9]

 



 Saturday, September 29, 2007

From .NET 1.x times, you might remember .manifest files as the way to enable XP visual styles support in your application.

Application manifests are important in Vista. If your application does not have a manifest, Vista will treat it as legacy application and enable file and registry virtualization. This allows the program to write to its own application directory even if it does not have write access to it - the writes are stored in some hidden folder in the user profile. This redirection can cause a lot of confusions when users try to save files using your program where they don't have write access, and instead of getting an error message the files get saved somewhere else!

Vista also has to determine if programs need administrative rights and should cause a UAC prompt. To do that, it uses a heuristic that looks at the file name and string resources to guess if it should elevate. Using a manifest, you can tell Vista if your program needs elevation and avoid the risk of Vista guessing incorrectly.

Previously, you had to distribute the .manifest file along with the .exe; or you had to embed it in the .exe using a post-build action calling mt.exe.

The C# 3.0 compiler has built-in support for embedding manifests, and actually creates a default manifest that turns virtualization off and tells Vista not to elevate if you don't explicitly tell the compiler not to embed a manifest.

Support for embedding manifests is now available in the SharpDevelop project options:

You can choose to embed the default manifest, not embed any manifest, or to embed a custom manifest.
If you select <Create...>, SharpDevelop will automatically create a file called "app.manifest" in your project. The app.manifest created by SharpDevelop is similar to the default manifest the C# compiler uses; it just adds some comments so you don't have to look up how to control Vista's UAC.

Categories: Daniel
Friday, September 28, 2007 11:08:15 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [1]

 



 Thursday, September 27, 2007

Yesterday I wrote about how I made compiling 30% faster by using multi-threading to let my dual-core CPU work with both cores. A problem was that I had to use serialize build events to send them back to the host process.

MSBuild provides these events to loggers (number of occurrences during the test shown in parenthesis):
MessageRaised   (21156)
ErrorRaised      (0)
WarningRaised   (18)
BuildStarted      (47)
BuildFinished     (47)
ProjectStarted  (449)
ProjectFinished (449)
TargetStarted (2910)
TargetFinished (2910)
TaskStarted (4308)
TaskFinished (4308)
CustomEventRaised (0)

All these events must be send to the SharpDevelop process, where logger AddIns can process them. Each one is serialized separately, and includes information about the full name of the project file being build and some other highly redundant information. Each event is about 1 KB in serialized form. Half of the events occurred in the worker running in the SharpDevelop process, but the other half occurred in the worker that ran in its own process; so half of the events had to be transferred, so we tranferred a total of 18 MB during the 25-second build. Remember the main cost is the serialization (.NET BinaryFormatter), not the data transfer itself.

Obviously, most of these events are not interesting for SharpDevelop, and the best way to improve performance is to ignore uninteresting events in the worker instead of ignoring them in the host.
So which events does SharpDevelop need? Potentially all, since AddIns can add loggers; but if those AddIns tell SharpDevelop which
events are required before the build starts, we can optimize for the common case that all AddIns are only interested in specific events - usually only in errors and warnings. Events that SharpDevelop currently does not use anywhere are MessageRaised, TargetStarted and TargetFinished - all are called quite frequently.
To our benchmark results from yesterday, I added an entry "Less events" where MessageRaised, TargetStarted and TargetFinished are deactivated:

Core Duo Notebook, 1 GB RAM, Vista Home Premium, Aero disabled:
 Rebuild:
   One worker:  35.2, 34.9, 35.2, 35.0, 34.3 : Average 35.1 seconds
   Two workers: 25.6, 23.9, 24.2, 23.5, 23.8 : Average 24.2 seconds
   Less events: 23.7, 23.5, 23.0, 22.4, 21.2 : Average 22.8 seconds

 Build (no changes):
   One worker:  7.4, 7.1, 7.2, 7.8, 7.1 : Average 7.3 seconds
   Two workers: 6.9, 6.8, 6.7, 6.6, 6.2 : Average 6.6 seconds
   Less events: 5.1, 5.0, 6.2, 5.6, 5.1 : Average 5.4 seconds

More than 1 second saved! (see yesterdays' post for details about the benchmark)

Now why does SharpDevelop need TaskStarted and TaskFinished? Take a look at the code:
void OnTaskStarted(object sender, TaskStartedEventArgs e)
{
  activeTaskName = e.TaskName;
  if (MSBuildEngine.CompileTaskNames.Contains(e.TaskName.ToLowerInvariant())) {
    AppendText("${res:MainWindow.CompilerMessages.CompileVerb} " + Path.GetFileNameWithoutExtension(e.ProjectFile));
  }
}

TaskStarted is just there to let the user know that building a project resulted in a recompilation - when doing a Rebuild, SharpDevelop
shows "Compiling ProjectName" for each project, whereas for a Build (no changes), no such lines are shown in the Output view.
But in fact, all consumers of TaskStarted probably look for specific task names. And all consumers of TaskFinished are also listening to TaskStarted and are also only interested in some specific types of tasks.
If we pass the build worker a list of task names we are interested in, it can send us only those events and we don't spend time serializing the tons of events about the tasks <Copy>, <Message> etc.
Together with the "Less events" changes, this decreases the event count from over 36000 to less than 1000.

I implemented this in revision 2696.

Now let's extend our benchmark:
Core Duo Notebook, 1 GB RAM, Vista Home Premium, Aero disabled, two workers, new event handling:
 Rebuild: 20.3, 20.9, 19.9, 20.2, 20.0 : Average 20.3 seconds
 Build (no changes): 4.3, 4.6, 5.0, 4.4, 4.2 : Average 4.5 seconds

That's better than "Less events" because we're sending even less events now, but logger AddIns can tell SharpDevelop that they need to receive additional events, so AddIns did not lose any functionality (but if you install such an AddIn, of course you would lose the performance benefits).
So we're now down from formerly 35.1 seconds (using only one core) to 20.3 seconds - that's 43% faster.
For "Build (no changes)", the reduction of the event count now makes using two workers efficient; from originally 7.3 seconds to 4.5 seconds is a 39% decrease in build time.

Categories: Daniel
Thursday, September 27, 2007 2:31:07 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [0]

 



 Wednesday, September 26, 2007

[Warning: long post. Short version: Building in SharpDevelop goes multi-threaded, scroll down for benchmark results]

SharpDevelop 3.0.0.2694 contains a rewritten build system. It has some new features that might be interesting for AddIn developers, but one new feature is directly useful for users, or more specifically, users with multi-core CPUs.

Here a quick history on build systems in SharpDevelop:
SharpDevelop 2.0 (the first version to use MSBuild), compiled solutions by telling MSBuild to build the .sln file.
SharpDevelop 2.1 and 2.2 compile solutions by telling MSBuild to build the projects individually.

When writing the build engine for SharpDevelop 2.1, I tried to make it multi-threaded so that two projects that are not dependent on each other can be built in parallel. However, this did not work: MSBuild makes use Environment.CurrentDirectory; and if two MSBuild instances run in the same process at the same time, they overwrite the CurrentDirectory and cause the other instance to search files in the wrong location.

Now I solved this problem by moving the build into worker processes. However, this means that all MSBuild events are serialized and remoted to the host process, where SharpDevelop AddIns need to have access to them - e.g. the CodeAnalysis AddIn registers itself as MSBuild logger, waits for the TaskStarted, ErrorRaised, WarningRaised and TaskFinished events and fixes certain FxCop error messages (some FxCop errors are not associated with a line number, but with a class/member name). Other SharpDevelop AddIns might decide that certain MSBuild events should be displayed in some form, e.g. in the output window. The only way to give AddIns access to all MSBuild events like they had before is to serialize all events and send them to the SharpDevelop process. When doing a build where there's relatively few work but lots of events (e.g. "Clean solution"), the worker is sending up to 1 MB/s of serialized events to the host.

This had a noticeable impact on performance, so I've kept the first worker in the SharpDevelop process, and only additional workers get their own process. So if you use only one worker, you won't see any worker process and should get the same build performance as SharpDevelop 2.x.

So even though building a project has multiple parts, of which some are waiting for the disk and others are waiting for the CPU; the overhead of using a worker process will cause the build to get slower. (MSBuild has to check if input files changed, start the compiler; the compiler will load input files, actually compile, write the output assembly; and some other stuff happens in the obj\ directory that you probably don't want to hear about)

But if you have a dual core processor, the CPU overhead doesn't hurt: a little overhead for one core is better than not using that core at all (this is assuming the compiler you use is not multithreaded).

But enough talking, let's get some benchmark results:

I used SharpDevelop 3.0.0.2694 to compile the source code for SharpDevelop 2.2.1:
AMD 3500+ (single core), 2 GB RAM, Vista Ultimate (32 bit), Areo enabled:
 Rebuild:
   One worker: 34.3, 34.1, 34.6, 34.7, 34.3 : Average 34.4 seconds
   Two workers: 39.6, 38.7, 38.1, 37.0, 36.6 : Average 38.0 seconds

 Build (no changes, so it's not compiling, but just checking everything for changes):
   One worker:  8.5,  8.0,  8.3,  8.0,  8.0 : Average  8.2 seconds
   Two workers: 13.4, 13.9, 14.0, 12.8, 13.1 : Average 13.4 seconds

The number of MSBuild events (Target started, Task started, Log Message...) is not much different between Rebuild and Build (no changes), so we get a constant overhead of about 5 seconds for the SharpDevelop solution on my desktop computer with a single core processor.

Here finally is the interesting part: Test results from my dual core notebook:

Core Duo T2300, 1 GB RAM, Vista Home Premium (32 bit), Aero disabled:
 Rebuild:
   One worker: 35.2, 34.9, 35.2, 35.0, 34.3 : Average 35.1 seconds
   Two workers: 25.6, 23.9, 24.2, 23.5, 23.8 : Average 24.2 seconds

 Build (no changes):
   One worker: 7.4, 7.1, 7.2, 7.8, 7.1 : Average 7.3 seconds
   Two workers: 6.9, 6.8, 6.7, 6.6, 6.2 : Average 6.6 seconds

For build (no changes), the overhead plays a big role, but using two workers still made the build a little faster.

But for the real job when there's something to compile, using two workers really rocks: My build is now 30% faster.

SharpDevelop 3.0 allows you to set the number of workers under Tools > Options > Project and solution > Number of projects to build in parallel. The default value for single-core processors is 1, for other processors it's 2. I don't know if going above 2 will have benefits for quad-core users as the overhead (both CPU for sending events to the host and memory because of the additional processes) increases with the number of workers.

Please leave a comment here with your test results - I'm looking for feedback from other multi-core users.

And if you're doing your own benchmark, some notes on how I did my benchmark:
- make sure enough RAM is free so that not only the system does not swap, but Windows has enough room to keep the files you are compiling in memory (this is generally a good idea if you want your system to be fast)
- don't count the first test run as the files are not yet in memory there
- reuse the same SharpDevelop instance for all test runs
- if you leave less than 60 seconds between test runs, SharpDevelop will re-use the worker process, otherwise it is shut down and has to be restarted. I did my tests re-using a single worker process. It might be interesting to see if the overhead of restarting the worker process is worth keeping it longer than 60 seconds.

Categories: Daniel
Wednesday, September 26, 2007 9:05:40 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [0]

 



 Friday, August 10, 2007

When you double-click a file in SharpDevelop 2.x, it will get opened inside SharpDevelop. AddIns can add display bindings opening for opening file types (e.g. ResourceEditor for .resx files), but this has the effect of disabling the other display bindings for that file type - In SharpDevelop 2.x, you cannot open a .resx file inside the text editor (except if you disable the ResourceEditor AddIn).

In SharpDevelop 3.0, you can now right-click a file in the project browser and select "Open With...". A dialog will show the list of display bindings that can handle the file type. There also is the entry "Use the default Windows application for this file" which will open the file as if you double-clicked it in Windows Explorer.

You can add new programs to the "Open With" list. SharpDevelop will store the list programs for each file extension in the SharpDevelop settings.

Here is a screenshot of the "Open With"/"Add program" dialog:

A very useful feature of the "Open With" dialog is the "Set as Default" button. This will mark an entry as the default program for opening files with the same extension in SharpDevelop. Double-clicks in the project browser will run the chosen default program. ("Set as Default" does not have any effect on double-clicks in Windows Explorer)

For example, SharpDevelop does not have a built-in image viewer/editor, so if you don't want the text editor, which is the fall-back for unknown file types, to open images, you can "Open With" a .png file in your project, enter your favorite image editor and set it as default program for .png.

Categories: Daniel
Friday, August 10, 2007 7:55:23 AM (W. Europe Standard Time, UTC+01:00)  #    Comments [0]

 



 Sunday, June 24, 2007

In revision 2583, I added the XamlBinding and WPF Designer to the setup of SharpDevelop 3.0.

XamlBinding is a language binding that integrates .xaml files into SharpDevelop's code-completion infrastructure.
Here is an example of SharpDevelop showing a tool tip for a property in a .xaml file:

A window in .xaml is a partial class: one part is generated from the .xaml file, the other is the user code. If the xaml file specifies a name for an element, the compiler will generate a field that can be used by the user code. XamlBinding adds code-completion support for such fields:

As you can see in the "Search Results" window, "Find references" now searches for references in .xaml files. The "rename" refactoring will update both the XAML and C# code.

Finally, the WPF designer that I wrote earlier this year is now integrated in SharpDevelop. Here is a screen shot:

The designer replaces SharpDevelop's property grid with its own, WPF-based property grid. This property grid will provide support for editing attached properties and data binding in the future.

Unlike the Windows Forms designer, which is included in the .NET Framework and merely hosted by SharpDevelop (which isn't as easy as it sounds); the WPF designer is written from scratch. Thankfully WPF makes this a lot easier than writing a Windows Forms designer from scratch would be.

The WPF designer is still missing lots of features; I would welcome any help.

You can download preview builds of SharpDevelop 3.0 from our build server.

Categories: Daniel
Sunday, June 24, 2007 4:23:00 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [5]

 



 Monday, May 14, 2007

Yesterday, I added a snippet parser to NRefactory. It tries to parse the input code as a compilation unit (full file containing class definitions), class body (=list of methods, properties, ...), statement list and expression. The result that produces the least number of syntax errors gets chosen.

This is useful for our online code converter, because often one wants to convert a simple code snippet and not a full file.

Here is the example code that converts the string "input" from C# to VB.NET:

SnippetParser parser = new SnippetParser(SupportedLanguage.CSharp);
INode node = parser.Parse(input);
// parser.Errors.ErrorOutput contains syntax errors, if any
// parser.Specials is the list of comments, preprocessor directives etc.
PreprocessingDirective.CSharpToVB(parser.Specials);
// Convert C# constructs to VB.NET:
node.AcceptVisitor(new CSharpConstructsVisitor(), null);
node.AcceptVisitor(new ToVBNetConvertVisitor(), null);
VBNetOutputVisitor output = new VBNetOutputVisitor();
using (SpecialNodesInserter.Install(parser.Specials, output)) {
  node.AcceptVisitor(output, null);
}
// output.Errors.ErrorOutput contains conversion errors, if any
// output.Text contains the converted code

Categories: Daniel
Monday, May 14, 2007 3:38:49 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [1]

 



 Thursday, November 09, 2006

ohloh.net is a service that analyses the source code of open source projects and creates statistics.

Click on this button to see the statictics for the SharpDevelop repository: 

All information on that page was automatically created by the service by looking at the subversion repository. It is wrong on the following points:

  • project age. Only the public repository of SharpDevelop 2.0 has been analysed. Earlier work on SharpDevelop 2.0, and the whole work on SharpDevelop 1.x and earlier beta versions is not included in the statistics. Old code taken over to the new repository was attributed to Mike Krüger.
  • License. It has found license headers in the source code of log4net and other libraries/tools included in the repository. That code is attributed to the person who added the library to SharpDevelop. The real license for SharpDevelop code (LGPL) is not found at all because we don't use the standard LGPL header.
  • The line count is slightly wrong for the same reason as above.
  • Only contributors having commit access are mentioned, patches are attributed to the person who checked in the patch.

On the contributors page, you can see activity graphs for the individual developers. And you can see the comment ratio, which unfortunately is quite low (far below C# average according to the "Factoids"), but other popular open source projects like Firefox, OpenOffice or MediaWiki seem to have the same problem.

Categories: Daniel
Thursday, November 09, 2006 10:38:22 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [0]

 



 Saturday, October 28, 2006

The SharpDevelop repository now contains a sample application that shows how to use our libraries ICSharpCode.SharpDevelop.Dom and ICSharpCode.TextEditor to enable code-completion.

You can find it in the folder SharpDevelop\samples\CSharpCodeCompletion. The expression resolver is part of SharpDevelop.Dom, so the example supports complex generic types etc. However it implements only code-completion when typing "." - no Ctrl+Space, keyword-completion or method insight. The code was kept as simple as possible, so multiple overloads are not filtered out, and there are no different icons for protected or internal members.

Categories: Daniel
Saturday, October 28, 2006 2:14:13 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [2]

 



 Tuesday, September 05, 2006

SharpDevelop 2.1 sports code-completion support for .NET 1.0, .NET 1.1 and the Compact Framework 2.0. In the "Compiling" option page of the project, you can choose the target framework:

Now code-completion is supported for all frameworks except the Compact Framework 1.0. Note that choosing Compact Framework is a new option in SharpDevelop 2.1 - previously, you had to modify your project file manually. If you did this, you had to change your project file manually back into the default .NET 2.0 one and then choose "Compact Framework 2.0" in the options - code-completion is available only this way. If you want to create a new Compact Framework project, there now also is a Compact Framework project template.

Note that code-completion requires the target framework to be installed on your machine - otherwise you'll get the default .NET 2.0 completion. For the Compact Framework, the Compact Framework SDK must be installed.

Categories: Daniel
Tuesday, September 05, 2006 7:31:08 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [1]

 



 Tuesday, May 02, 2006

I know this isn't related to SharpDevelop; but I still wanted to let you know:

The Baltic Olympiad in Informatics will take place in Heinola, Finland, May 18th to 22nd. Look at the  list of participants - I'm part of the German team!

I'll have to learn algorithms for "usual" problems (geometrical algorithms, graph algorithms; dynamic programming). If you know any interesting solutions (read: non-standard, I already know the "classics" like Booyer Moore, MST, maximum flow) to them (preferrably for C++/STL), please leave a comment!

Categories: Daniel
Tuesday, May 02, 2006 6:51:19 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [2]

 



 Saturday, March 04, 2006

Today I wrote a little program in Boo that can analyze the results of Subversion's "blame" command. "blame" goes through the whole history of the subversion repository and finds for each line of code the revision in which it was written.

I analyzed the trunk of our current repository for SharpDevelop 2.1. The repository was created on the 4th January of 2005 for SharpDevelop 2.0. Development on SharpDevelop 2.0 was already active before, but that old repository is not available anymore. That means I could not find out who wrote code older than 2005 - most of the code marked as "unknown old code" is probably from SharpDevelop 1.0, but all the changes done by Mike Krüger to make SharpDevelop run on .NET 2.0 using MSBuild for the project system are also "old code". Our parser library NRefactory was existant at that time, too.

My analyzer program gets the person who committed each line of code. Additionally, it searches log messages for the term "patch by" and uses that name instead. And I assigned some revisions manually, for example when I committed Peter Forstmeier's SharpReport. Moreover, I excluded the docking library and log4net (the source code for both is included in our repository).

Now here is the image with the results:

These results were pretty surprising for me. Considering that "unknown (old code)" already ran on .NET 2.0 and already had the improved AddIn system and build system in place (all those changes were done by Mike Krüger), pretty much all of SharpDevelop has been rewritten. The area with the highest percentage of old code is the text editor - 80 percent are unchanged. Most other parts are around 45% old code - the 33% average is caused by the new AddIns.

"unknown (merged code)" is code that has been committed to SharpDevelop 2.0 in the last time and then was merged back into trunk (SharpDevelop 2.1). Most of it seems to be David Srbecky's generated debugger interop code.

Finally, sometimes I committed patches and my own modifications in the same commit; everything was counted for the person providing the patch. For example, Christian Hornung owes me 170 lines on the AddIn manager.

So the whole thing isn't really accurate, but if you are interested in the source code, you can download it here:
BlameAnalyzer.zip (1,1 MB)  Caution: contains hard-coded absolute paths to a temp directory and the SD working copy

Categories: Daniel
Saturday, March 04, 2006 10:21:13 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [0]

 



 Wednesday, February 15, 2006

If you're building SharpDevelop from our Subversion repository, you might already have noticed it: The trunk of the repository now contains SharpDevelop 2.1 (Codename "Serralongue").

Changes in the 2.1 line so far: The Subversion AddIn was reactivated (note that it does not work on x64), a Boo interpreter pad was added, the docking library was updated and now supports VS2005-style "dock helpers", and you can now choose the target framework version for VB projects.

Our wiki gives more information on how to get the code from the repository; the build server will build the 2.0 line only for the moment.

If you are already using the repository but want to stay on the 2.0 line, you will have to switch your working copy using this command:
svn switch svn://sharpdevelop.net/sharpdevelop/branches/2.0/SharpDevelop

Bug fixes are committed to the 2.0 branch only and every week they will be merged back to 2.1. New features will be added to 2.1 only.

Categories: Daniel
Wednesday, February 15, 2006 4:15:47 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [0]

 



 Wednesday, January 04, 2006

I've published a CodeProject article about ICSharpCode.Core. It demonstrates how you can use the SharpDevelop Core to create your own applications. Link: http://www.codeproject.com/csharp/ICSharpCodeCore.asp

Categories: Daniel
Wednesday, January 04, 2006 2:17:22 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [1]

 



 Monday, December 26, 2005

This video shows what you need to make your AddIns ready for the AddIn Manager and package them as ".sdaddin"-files.

AddInManager.wmv (2,37 MB; 05:12 m)

It uses the Image Viewer addin from the third AddIn-writing tutorial.

Additional downloads:
MyTestAddIn.AddInManager.zip (4,53 KB) - source code of the test AddIn
ImageViewer.sdaddin.zip (2,92 KB) - binary package of the test addin (remove the ".zip" extension, it's just there because this blog doesn't accept ".sdaddin" files)

There is a bug SharpDevelop 2 Beta 1 which causes the AddInManager to crash when opening the .sdaddin-file created in this video. It has been fixed in build 940, so make sure you get a recent build from the build server.

Categories: Daniel
Monday, December 26, 2005 9:18:32 PM (W. Europe Standard Time, UTC+01:00)  #    Comments [0]