Microsoft Versus
Dissecting Microsoft | Directory

Microsoft Versus Open Source Software Bugs

"There are no significant bugs in our released software that any significant number of users want fixed." - Bill Gates (Focus Magazine, nr.43, pages 206-212, 23 Oct 1995)

Both the closed Microsoft® Windows™ platform and the open Linux™ platform contain bugs. Discovery and discloser of those bugs and the availability of fixes differ greatly. Bugs and availability and installation of patches have a huge impact on long term [cost of ownership].

Comparing Closed and Open Source Bugs

It's in the best interest of anyone who profits from creating software to keep problems with that software a secret. Publicly aknowledging all of the problems with a software package can make that software appear inferior. However, being caught not disclosing all known problems can be a public relations nightmare. Proprietary software vendors must therefore find a balance between publicising their problems and only taking care of them internally. They can, for example, release a patch which fixes issues which were never publicised, which shows they are working on their problems while not explicitly stating what those problems are or how many users they have affected. The public actually has no way to know if a company is fully disclosing all issues of which they are aware. With closed source software few people except for the creator are able to look through the source code to find bugs, requiring anyone else to explicitly use every feature of the software in every way possible to find any issues. And generally the public relies on these companies to use their internal testing resources to find every issue. Microsoft's official policy is to not disclose details about [security] related bugs until a patch is available, leaving the possibility that there are more bugs known to Microsoft than to the public.

With proprietary software, usually the reputation of the company is on the line, not any individual developer. Since developers are expected to make occasional bugs, they generally do not lose pay compensation for these mistakes. While closed source developers do not want to make mistakes, they typically have little to lose if they do. Their motivation, and also the strength of that motivation, is different than for open source developers.

[Open source] projects, on the other hand, have no way to hide behind closed doors. Their code is available and their software is usually free. Publicising the code means anyone can pick up the code and look for errors or run the application. In fact, while software vendors rely primarily on hired or registered testers, free and open source software developers rely on the public to test their software. No registration, pre-approval, or payment is required. The fact that more eyes can look at the code helps ensure more bugs are found and none can ever be hidden from the public. It's in the best interest of open source projects, and their users, to have bugs found and fixed as quickly as possible. Most open source developers are working for their reputation, which relies on the quality of their code. Since their motivation is intellectual and emotional and not financial, they have a bigger incentive to fix every possible bug.

For the reason of closed source secrecy it is impossible to accurately compare bugs and related security vulnerabilities between most closed and open source software. Intentions and motivations differ. Abilties of common users often also differ, causing separate groups to find bugs in each set of software in different ways. There is no generally accurate way to compare how well written two software packages are written when one's source code is available and the other is not. The only possible comparison, therefore, is from user experience. When taken with a narrow focus, software can be compared by users and the user can decide which is best for her/his situation. General comparisons about entire platforms are less useful since they are less accurate, but will continue to be made in attempts to make intelligent choices. Each fact should be taken at only face value, with not too much read into how it compares to other platforms since the intentions, motivations, and methods are different for each.

Microsoft® Windows™ Bugs and Patches

Microsoft® bug counts are not publicly published. They are found through accidental leaking of information on their web site or from internal memos. Windows distributions contain a minimal set of workstation and administrative applications. Windows NT 4 contained no server applications. Windows 2000 contained one server application, IIS. Neither contain compilers or development tools.

It's become common to delay use of the first release of Microsoft software due to expected bugs. With hundreds of thousands of beta testers the existance of many, if not most, bugs are known prior to release. Releases aren't usually delayed to allow fixing of all bugs. Microsoft prefers to release the version available at the time and let users download fixes later. Home users and corporate support personnel spend many thousands of hours installing fixes to Microsoft software throughout each year. Many claim it's become nearly impossible to keep up with the quantity of patches released. Microsoft themselves have not been able to install all of their critical patches to software they run themselves, leaving [security] vulnerabilities open for worms to spread within their own corporate network.

Many thousands of bugs exist Microsoft software which will never be fixed. There remain many users of Microsoft Windows NT 4, for which no more patches will be released. Looking up bugs on Microsoft's web site, however, reveals many issues which will never be given a final resolution. Of course it's not practical for a company to hold themselves accountable for all past versions of their software forever. However there is a need to support a any large base of users. Requiring the purchase of more software is not a valid solution.

Open Source Bugs and Patches

Free and [open source] software (FOSS) counters this problem with continually updated releases. Each component of a complete system is continually patched as needed, allowing customers to patch only the software which requires it, if desired. Unlike Microsoft's large patches, small ones are usually available which will resolve issues without introducing new ones.

Scientific research has proven the open source model to be more effective in removing bugs from software, even when the quality of the programmers is lower than in a closed system. The research "shows that closed-source projects are always slower to converge to a bug-free state than bazaar open-source projects."

GNU/Linux

Most GNU/Linux distributions publicly display bug statistics, Debian being the most comprehensive. Most GNU/Linux distributions contain the OS plus every kind of server and workstation software ever likely to run on it (e.g. multiple “office suites”, two desktop environments, industry standard compilers, development tools, PGP encryption, clients and servers for e-mail, HTTP, FTP, etc.). This is a very important distinction when performing comparisons.

Windows NT 4



Windows 2000



Debian GNU/Linux - Development version

Around the time of Windows 2000 release (mid-Feb. 2000)



Bug Statistics Conclusions

GNU/Linux distributions contain sufficient applications for operating a workstation and server, yet contain fewer bugs than any Windows OS not containing required applications.

Being proprietary software, Windows bugs are counted as those reported by beta testers and Microsoft before software release, plus end users, security firms, and Microsoft after software release. GNU/Linux bugs, having source code available to all, are reported by developers, alpha/beta testers, end users, and security firms before and after release of final software versions.
Copyright © 2004-2007 Matthew Schwartz