CONCLUSIONS

This study has confirmed that the Mono runtime is a reasonably mature and practical alternative to Microsoft's .NET Framework for running Web-based, n-Tier, distributed applications. Code developed on Windows using Microsoft tools runs without alteration under Mono on Linux although developers need to be aware of a few differences between the Windows and Linux platforms that require code to be written in a way that accounts for these differences, such as:

• File system paths should not be hard coded
• Services should have their user property set to LocalSystem at compile time

It is, in the author's view, remarkable that so little has to be done to deploy code at this level of sophistication, developed on a Windows platform, using Microsoft tools to what is, in many respects, a foreign environment. Before going on to comment on the performance aspects it is perhaps worth remembering that, as someone (possibly Oscar Wilde, but I have been unable to trace the source) once said on seeing a dog dancing:

"One is not so much amazed that it dances well, but that it dances at all."

Because of the lack of fully functional, mature development tools for .NET application development on Linux, it is likely that most cross-platform code will be developed on Windows and deployed to both platforms. This is especially true in the field of ASP.NET development where there is really nothing available other than for a Windows environment. The MonoDevelop IDE shows great promise and will eventually provide at least an adequate development platform for Linux and other platforms. It will be some time, however, before it is as feature-rich as Microsoft's Visual Studio. It is important, therefore, that code developed in this way performs adequately when deployed to other platforms. The results of the Web-based application experiment demonstrate clearly that interactive applications of this type will probably not be penalised by running on the Mono virtual machine on Linux. There are too many other latencies that affect program performance for compiler or runtime differences to be statistically significant. It is even less likely that there would be any performance differences that a user could perceive.

Nevertheless, the automatic upload experiment and the other performance test data does indicate that there are some quite significant differences between Mono and the .NET Framework. Such tests are not, of course, meant to be realistic. How many times would a developer be asked to write code to upload over one thousand XML records one million times? Not very often in all probability. But this is not a reason for ignoring those differences that do exist.

The indications are that the major difference is in the performance of the two runtime environments. The findings are by no means conclusive. Anghel and Tertin found little in the way of significant difference, whilst [Marcus] found an advantage in favour of the .NET Framework runtime in three out of five tests. The tests in this study found significant differences in favour of the .NET Framework in four out of the five tests. Interestingly Tertin, Marcus and this study all find the Mono runtime engine to be superior at dealing with exceptions.

Although not conclusive, the project's sponsor has expressed some satisfaction with its results:

"In completing this project, Peter provides some specific examples on how the challenges of compatibility can be addressed, when attempting to apply architecture level development principles. Through his benchmarking and producing statistical analysis of differences, he shows the risks associated with accepting 'carte blanche' the development vendors' claims of complete compatibility. I find his project very beneficial in terms of laying out an approach to assess such claims; I will be sharing the results with my colleagues, who are currently working on providing a systematic method for applying standard object oriented programming constructs across multiple platforms."

Only this study has made a direct comparison between the same byte code running on the .NET Framework runtime engine on Windows and running on the Mono runtime under Linux.

Since it is likely that the most common scenario will be the one tested here - code compiled using Microsoft tools on Windows being used across platforms - the study indicates that there may be a need for the Mono team to look at the reasons why it should be that this code tends to perform relatively poorly on the Mono runtime.

It is not clear from this project, of course, to what extent the differences observed in the automatic upload test between the two environments were due to differences in the operating systems. However since the results appear in many ways to agree with results obtained on the Windows platform only, it is likely that the major component here is the runtime engine itself.

At the very least this study indicates the need for further work to be carried out to compare the two environments. One of the difficulties is that there are a very large number of variables to consider.

It is the author's view that the value of Mono at the moment is in providing a cross-platform runtime environment. It is doubtful that this is what the Mono team would want to hear, but the chances of Mono becoming the compiler and runtime engine of choice on the Windows platform seem remote to say the least. It is also unlikely at present that developers will choose to develop on Linux or some other platform and then deploy to Windows, at least until development tools of equal sophistication to those available on Windows are available for those other platforms.

Given this, the fact that Web-based, n-Tier, distributed applications are likely to run without any noticeable performance penalty on the Linux platform is extremely encouraging. However it should be noted that this application measured less than 150 instances over a period of 14 days. Further work needs to be done to assess whether performance suffers, on either platform, when there are multiple simultaneous hits.

The fact that non-interactive code compiled with csc seems to run slower on the Mono runtime engine is a cause for concern. Useful work could be done here developing a standard test suite for use in comparing the performance of the two runtime engines.