On this page...
|| Monday, August 17, 2009
In SharpDevelop 126.96.36.19911, I have rewritten the ParserService class.
There are lots of changes.
First, from the point of view of an AddIn implementing IParser:
- SharpDevelop will not create a single instance of your class, but one instance per file.
- Keep in mind that there might be concurrent parser runs for the same file. IParser implementations must be thread-safe.
- The ITextBuffer interface now provides a 'Version' property which allows comparing two versions of the same document and efficiently retrieving the changes between them.
Together, these changes allow for the implementation of incremental parsers. The parser instance can simply store the ITextBufferVersion of a the last run in a field and use it to detect the changes to the next version.
We do not plan to write replace our existing parsers with incremental parsers now - but we are working on an incremental XML parser. The XAML code completion support is already taking advantage of this; and once we re-implement XML code folding for SharpDevelop 4, it will likely use this incremental parser, too.
However, a Parser instance should never cache the ICompilationUnit or parts of it: it now is possible for a file to have multiple compilation units (one per project that contains it). See "Support for files shared between multiple projects" below.
Originally, I wanted to give a "no-concurrency guarantee" for IParser implementations, i.e. the ParserService would ensure that there is only a single parser run for each file. However, to implement this, a
per-file lock while calling into the parser was required. The main thread could wait for existing parser runs to finish while the parser implementation would wait for the main thread to run an invoked method -> deadlock.
In the end, I decided the IParser implementation should have the responsibility for this - if it needs to avoid concurrent execution, it should use a lock.
For someone using the ParserService:
- Methods dealing with assembly references have been moved into the new class 'AssemblyParserService'.
- The remaining methods are now documented, in particular regarding their thread-safety.
- All events are now raised on the main thread. This guarantees that events arrive in the correct order and makes consuming them easier.
- EnqueueForParsing has been renamed to BeginParse and now provides a future (Task<ParseInformation>) to allow waiting for the result. However, to avoid deadlocks, this should not be done by any thread the parser might be waiting for (especially the main thread).
- The ParseFile method does not necessarily parse the snapshot of the file you specify - it might parse a newer version instead (but never an older version). Unlike BeginParse().Wait(), ParseFile() is safe to call from the main thread.
- If a file hasn't changed, calling ParseFile is a no-op.
- The ParseInformation class has been made immutable. Support for 'ValidCompilationUnit' and 'DirtyCompilationUnit' has been removed.
- The GetParser() method allows retrieving the IParser instance for a specific file. This is useful in some special cases for using details of specific IParser implementations.
The API changes here are more limited. Most important is the change to ParseInformation: the existing concept of keeping an old but valid compilation unit during parse errors was dropped because the was no useful upper bound on the age of the valid compilation unit; in some cases the 'valid compilation unit' might be several hours old and would represent an empty file.
All CompilationUnit-properties on ParseInformation now return same value; the old properties will be marked [Obsolete].
If a parser wants to reuse information from old parse runs because its error recovery is not reliable enough, the parser itself now has to maintain this state and mix it into the new compilation units - doing this is much easier now due to 'one IParser instance per file'.Support for files shared between multiple projects
Also, there has been a major internal change that isn't apparent in the API:
A single file can now have multiple ParseInformation instances - one per project that contains the file. Previously, files used by multiple projects would show up in code completion only for one of the projects. Now the file will be parsed once for each project that contains it.
Because a single IParser instance is used for all these parse runs, it is possible for incremental parsers to avoid redundantly parsing the file. However, a separate ICompilationUnit must be produced for each run because it contains a pointer to the parent project content.
|| Saturday, June 13, 2009
SharpDevelop uses the MSBuild libraries for compilation. But when you compile a project inside SharpDevelop, there's more going on than a simple call to MSBuild.
SharpDevelop does not pass the whole solution to MSBuild, but performs one build for each project. This is done to give SharpDevelop more control - e.g. we can pass properties to MSBuild per-project. For example, when you click "Run code analysis on project X", we'll first build all dependencies of X normally, and then project X with code analysis enabled. A normal MSBuild call (e.g. if you use it on the command line) would perform code analysis on all dependencies of X, too.
When calling MSBuild on a project on the command line, MSBuild will find all referenced projects and build them recursively. We have to prevent this kind of recursive build inside SharpDevelop, as we already took care of the referenced projects. We don't want MSBuild to check referenced projects for changes repeatedly (this would dramatically slow down builds), and of course we need to prevent MSBuild from running stuff like code analysis on the dependencies. Fortunately, the Visual Studio team had the same requirement, so SharpDevelop can simply set the property "BuildingInsideVisualStudio" to true to disable recursive builds.
However, using that property, MSBuild will call the C# compiler on every build, even if no source files were changed. This is desired in Visual Studio - VS has its own "host compiler" that does change detection. But it's bad for SharpDevelop. As a workaround, we override the MSBuild target responsible for this and fix it. This is an in-memory modification
to your project file; it never gets saved to disk.
Another point where we use this kind of in-memory modifications is for importing additional .targets files
in your project. Some AddIns in SharpDevelop do this to add new features to the build process - for example the code analysis AddIn.
Now enter parallel builds
. It would be nice to be able to call MSBuild on multiple threads and compile projects in parallel. Unfortunately, that's not possible. MSBuild uses the process's working directory
as a global variable. In one process, only one build can run at a time. Even worse: if you have MSBuild in your process, all your other code must deal with concurrent changes to the working directory.
In .NET 3.5
, Microsoft introduced the "/m" switch in MSBuild. This makes MSBuild create multiple worker processes (usually one per processor), enabling concurrent builds. Unfortunately, this feature is exposed in the MSBuild API only through a single method, and that only allows several project files from the hard disk in parallel. It does not support in-memory projects; it cannot even compile projects in parallel if there are dependencies. Microsoft solves the latter problem by separating the project in the solution into 'levels' which depend only on projects from the previous level. However, this doesn't mix well with the way building is integrated in SharpDevelop - we don't use levels but do a kind of topological sort on the dependency graph, and not all SharpDevelop project have to use MSBuild; AddIn authors could choose their own project format and build engine. In the end, I had to create my own build worker executable
in .NET 4.0
, Microsoft created a completely new MSBuild API
. The 'level' problem is solved: the new API allows adding new jobs to a running build. It also seems like it is possible to build in-memory projects in parallel now. But as it turned out, in-memory changes only work in the primary build worker (in-process), all other workers load the file from the hard disk and ignore our changes
Instead of going back to our custom build worker, however; I decided to find a different solution for handling our modifications. Microsoft.Common.targets contains several extensions points adding custom .targets files by setting a property. A good solution for added a custom new target might be setting "CustomAfterMicrosoftCommonTargets" to the name of the .targets file. However, this might conflict with projects that already use this feature, so instead I chose "CodeAnalysisTargets". SharpDevelop comes with its own code analysis targets, so it doesn't hurt if we disable the Microsoft targets.
So in the end, the solution is trivial: create a temporary file containing only our modifications and set a property to tell MSBuild to pick up that file.
Why couldn't I simply write the project file including the modifications into a temporary file? The MSBuild reserved properties
would point to the temporary file and custom build events using those properties would likely fail.
|| Friday, May 22, 2009
As you've probably already heard, Microsoft released .NET 4.0 Beta 1 on May, 20th.
SharpDevelop 4.0 will be the SharpDevelop version built on top of .NET 4.0. I've just got it running:
Yes, that message really reads: "Compiling is not yet implemented". There were huge changes in MSBuild 4.0 and the parts of SharpDevelop's project system that are talking to MSBuild will have to be rewritten.
The .NET 4.0 work on SharpDevelop is going on in the dotnet4 branch, for which we do not provide builds. The dotnet4 branch will be merged back into trunk as soon as it's good enough so that I think other SharpDevelop contributors can be expected to use it.
But myself, I'm currently stuck using a dysfunctional version of SharpDevelop for development. I cannot use previous versions since they cannot compile for .NET 4.0 and don't have code completion for 4.0 libraries. Using the dotnet4 version of SharpDevelop at least gives me code completion for 4.0 libraries, but it looks like I'll have to compile from the command line for some time.
I cannot even use Visual Studio 2010 as it doesn't want to open ICSharpCode.SharpDevelop.csproj - it says it's an unsupported project format. Even if I recreate that project in Visual Studio, VS doesn't want to open it - looks like a VS bug to me.
|| Thursday, May 14, 2009
It is finally available
The really newsworthy feature is the profiler - it has been in the works for over a year, and it is the first full-fledged open source profiler for .NET. Siegfried would love to hear your feedback in the forums, especially which features should be next in the pipeline.
|| Saturday, April 04, 2009
The time to send in proposals for Google Summer Of Code is over now.
Now we're busy reading your proposals and trying to decide on a ranking. This is a lot more work than I initially expected - we got lots of proposals during the last three days. Unfortunately, most of the late proposals were of a rather low quality.
In total, we got 44 proposals from 34 students - much more than I expected.
Here's the list of topics proposals were written on. As you can see, most of them come straight from the ideas page.
- 10 proposals on Database tools
- 5 class diagram / UML related
- 4 Edit and Continue / C# background compilation
- 4 ASP.NET
- 4 Refactoring
- 3 C++ support
- 3 Debugger visualizer
- 3 Customizable Shortcuts
- 1 VB 9 code completion
- 1 Pretty printer
- 1 XAML code completion
- 1 Integrated bug tracking
- 1 actually creative idea
- 1 idea completely unrelated to SharpDevelop
- 1 idea I couldn't understand - due to completely broken English and an empty 'Details' section
- 1 proposal that didn't have any idea
But we're looking for students who would like to join the SharpDevelop team; we don't simply want to get some work done. So it's possible that we'll pick multiple students from the same 'category'; and having the only proposal on a much required feature doesn't mean you're automatically accepted.
There also were some Java proposals but I'm not sure where they disappeared to. In any case, SharpDevelop is a .NET IDE, not a Java one. There are already good open source Java IDEs available; no need to add Java support to SharpDevelop.
This is our first GSOC and I'm not too sure how we should judge the proposals. A surprisingly large part of them is obviously disqualified because the proposal is missing necessary details / the template isn't filled out completely. And what to do with a student who makes a promising impression but chose a project that isn't really interesting to us; or looks like it's not enough work for GSOC? What about projects that look like they cannot be done in the GSOC time frame; but it might be possible for a good coder and the Bio looks like the student knows what he's doing?
We don't know yet how many slots Google will give to us, so we are as excited as you are :)
|| Friday, April 03, 2009
In SharpDevelop 188.8.131.5248, I changed our Subversion integration to use SharpSVN instead of SvnDotNet.
SharpSVN exposes more Subversion APIs to managed code, which could result in some nice features in the (far) future - for example, "SVN Diff" right inside the text editor.
But the main reason for the upgrade was that SharpSVN supports Subversion 1.6. If you are using TortoiseSVN 1.6, you need to update to SharpDevelop 3.1. The old SvnDotNet does not work with new working copies.
However, the same is true in the other direction: if you use SharpDevelop 3.1, you must update to TortoiseSVN 1.6. No matter which .NET wrapper or client version is accessing a repository, the underlying Subversion library has the unpleasant feature to automatically upgrade working copies. As soon as the Subversion 1.6 library inside SharpDevelop touches your working copy, Subversion 1.5 clients will no longer be able to access it.
You need to update all Subversion clients on your machine at the same time. SharpDevelop contains a Subversion client:
- SharpDevelop 3.0 comes with Subversion 1.5 and requires TortoiseSVN 1.5.
- SharpDevelop 3.1 (starting with revision 3948) comes with Subversion 1.6 and requires TortoiseSVN 1.6.
This entry in the Subversion FAQ describes the problem and offers a working copy downgrade script, in case you decide to go back to a previous SVN client version.
|| Tuesday, March 24, 2009
David wrote a blog post on our participation in the GSoC 2009. It refers to the GSoC 2009 Wiki entry where the mentors have posted a few ideas (yes, we actively encourage students to come up with their own!). As I am the program manager, I am not active as a mentor, but I sure do have an opinion... here are my top five projects when it comes to "visibility" (a feature that will be used by a large percentage of our user base):
- The database scout / database api / EDM designer (we already have a student lined up for this)
- Debugger improvements (visualizers, edit and continue support, ...)
- VB.NET 9 support on par with our existing C# 3.0 support (most likely a task for a CS student)
- Refactoring support (think ReSharper)
- Customizable keyboard shortcuts / toolbars
I took "Everything .NET 4.0" from my top five because Beta 1 of Framework 4.0 most likely won't hit the streets before the student application deadline. And I will post further ideas in David's blog post, so you might find something palatable there too.
|| Tuesday, February 10, 2009
Another major release of SharpDevelop finally hits the streets - version 3.0 has been completed yesterday and is available for download as of now! Go get it
|| Sunday, December 14, 2008
On December 5th (revision 3655), we removed the WPF designer from
SharpDevelop 3.0. The decision to remove this feature had its origins
in an internal discussion between Daniel, Ivan and myself after watching the PDC
2008 session Microsoft .NET Framework: Declarative Programming Using
XAML. There, System.Xaml.dll was announced - a parser for the full Xaml standard.
Back then, a preview was announced for November, but that has been pushed back - please see the System.XAML CTP blog post by Rob Relyea. Therefore we had to take the decision whether to use System.Xaml.dll without doing a prototype, but we opted for saving ourselves the chore of having to keep abreast with future XAML changes by sticking with a standard parser instead of having to maintain our own.
Because System.Xaml.dll is part of .NET Framework 4.0, we had to remove the WPF designer feature from SharpDevelop 3.0.
|| Saturday, December 06, 2008
In version 184.108.40.20660, I changed how NRefactory's TypeReference.Type works. This is a potentially breaking change to users of NRefactory.
Previously, TypeReference had both a "Type" and a "SystemType". If you parsed "long a;", you got a TypeReference with Type="long" and SystemType="System.Int64".
However, this got a little problematic if you wanted to modify the AST - do you have to change both Type and SystemType? Actually, setting Type to "int" was automatically setting SystemType to "System.Int32" - even if you were modifying VB code which doesn't have the "int" keyword but uses "Integer". The other way round, setting Type to "DATE" would set SystemType to "System.DateTime" - not only for VB, but also for C#. Because the parser internally also uses the Type setter, "DATE d;" would parse to "System.DateTime d;" in C#!
To solve this, I removed the language-specific "Type". Now the Type property always contains the SystemType. You can use the new boolean IsKeyword property to tell if the type was specified using the language keyword or if the CLR type was specified explicitly.
Another related problem was that there was no way to output "System.Int32 a;" using NRefactory - the output visitor would always automatically convert it to "int a;". I changed this, too - now the output visitors will use the short form only if the IsKeyword property is set. So code generators using NRefactory will output the long form when using the new NRefactory version unless they are modified to set IsKeyword=true.
© Copyright 2016 SharpDevelop Core Team