|
Estimating the Total Development Cost of a Linux Distribution By Amanda McPherson, Brian Proffitt, and Ron Hale-Evans October 2008
The Linux operating system is the most successful open source project in history, but just how much is the software in a Linux distribution “worth”? In 2002, David A. Wheeler published a well-regarded study that examined the Software Lines of Code present in a typical Linux distribution. His findings? The total development cost represented in a typical Linux distribution was $1.2 billion. We’ve used his tools and method to update these findings. Using the same tools, we estimate that it would take approximately $10.8 billion to build the Fedora 9 distribution in today’s dollars, with today’s software development costs. Additionally, it would take $1.4 billion to develop the Linux kernel alone. This paper outlines our technique and highlights the latest costs of developing Linux. The Linux operating system is the most popular open source operating system in computing today, representing a $25 billion ecosystem in 2008.1 Since its inception in 1991, it has grown to become a force in computing, powering everything from the New York Stock Exchange to mobile phones to supercomputers to consumer devices. As an open operating system, Linux is developed collaboratively, meaning no one company is solely responsible for its development or ongoing support. Companies participating in the Linux economy share research and development costs with their partners and competitors. This spreading of development burden amongst individuals and companies has resulted in a large and efficient ecosystem and unheralded software innovation. Over 1,000 developers, from at least 100 different companies, contribute to every kernel release. In the past two years alone, over 3,200 developers from 200 companies have contributed to the kernel.2 It’s important to note that the kernel is just one small piece of a Linux distribution. A distribution is actually made up of multiple components including the kernel, the GNOME and KDE desktop environments, the GNU components, the X window system, and many more. The total of individual developers contributing to these projects surely numbers in the thousands. Because Linux has been developed collectively, there is no single source for cost estimates of how much it has taken to develop the technology. In 2002, David A. Wheeler published a well-regarded study that examined the Software Lines of Code present in a typical Linux distribution (Red Hat Linux 7.1)3. He concluded—as we did—that Software Lines of Code is the most practical method to determine open source software value since it focuses on the end result and not on per-company or per-developer estimates.4 Using the industry standard tools he developed to count and analyze SLOC, he determined that it would cost over $1.2 billion to develop a Linux distribution by conventional proprietary means in the U.S. That was six years ago. Since Linux is innovating and growing at a high rate year over year, the $1.2 billion number needed to be updated to reflect the true value of development reflected in Linux today (and the rising cost of software development itself). For this paper, the Linux Foundation set out to determine the total development cost represented in a typical Linux distribution and to update that $1.2 billion number widely used since its publication in 2002. We analyzed the Fedora 9 distribution, which was released on May 13, 2008. It is a popular and well-used distribution and is the base for Red Hat Enterprise Linux which represents a large percentage of the Linux market. It is also a direct descendant of the Red Hat Linux 7.1 software analyzed by Wheeler in his original paper. For this study, we used David A. Wheeler’s well-known SLOC tool, SLOCCount. SLOCCount makes use of the industry standard COnstructive COst MOdel (COCOMO), an algorithmic Software Cost Estimation Model5 developed by Barry Boehm6. The model uses a basic regression7 formula, with parameters that are derived from historical project data and current project characteristics.8 We updated his study from 2002 to include the growing code base of the Linux kernel and other packages as well as the higher annual salary of a software developer. (More detail on this follows in the Approach section of the paper.) Using this approach, we estimate that it would take $10.8 billion to develop the Linux distribution Fedora 9 by traditional proprietary means in year 2008 dollars.
Approach Our basic approach was to:
For those who are interested in how we installed and analyzed the source code, please see the Appendix. To update the 2002 study, it was decided that we would use the same base code used in the 2002 study, Fedora—the community distribution that forms the basis of Red Hat Enterprise Linux. For our examination, we decided to use Fedora 9. We counted all the available Fedora 9 packages that were made public in the mirrors.kernel.org mirror archive. It was decided to use all of the packages because we did not wish to use an arbitrary definition of which flavor of Fedora 9 we would measure. Fedora contains significantly more software than Red Hat’s enterprise version. One reason for this is a diverse community is involved in building Fedora, not just a company. Using the SLOCCount application is a remarkably simple task: it is simply a matter of pointing it at the correct directory where the source code resides, and letting it run. A detailed explanation of how the program works and how to use it is still available at Wheeler’s website9. To calculate the costs for these distributions, a base salary was found for computer programmers from the US Bureau of Labor Statistics. According to the BLS, the average salary for a US programmer in July, 2008 was $75,662.0810. This was the salary amount used in our SLOCCount run. (Of course most software development these days is global so using a US-only salary number is somewhat specious. One avenue of continued exploration would be to determine a global average salary to use as a baseline for production costs.) It should be noted that salary alone is not the sole determinant of software development costs. In his articles, Wheeler notes the presence of the overhead parameter within SLOCCount. This takes into account the costs of testing, equipment, company operating costs, and total compensation for the developer. This parameter is also known as the wrap rate. As just one example of how these wrap rate costs can quickly increase, in the United States, the average compensation percentage (sick days, vacation, insurance benefits) for a professional employee is 29.0 percent11. Thus, the programmer making the average US salary figure of $75,662.08 is actually costing the employer $97,604.08 in compensation alone. This is just one piece of the total wrap rate pie. Wheeler’s own estimate of the total wrap rate value is 2.4. Though this figure clearly varies by company and region, further research did not show any significant evidence that this figure needed adjustment for our tests.
Source Code and Estimated Cost Results Finally, given all the assumptions shown previously, the SLOC and estimated production values for Fedora 9 are as follows.
As an aside: the original SLOCCount annual salary figure was $56,286 in year 2000 dollars, so it’s of interest to see just how today’s costs translate to those same 2000 costs. To find this, we multiplied the total cost to develop Fedora 9 in 2008 dollars ($10,784,484,309) by 0.744, which is the ratio between the year 2000 programmer’s salary normally used by SLOCCount ($56,286) and the July 2008 salary we found ($75,662.08). The result is just over $8 billion to develop Fedora 9 in year 2000 costs. This gives the reader a good indication of just how much code and functionality has been added to a Linux distribution in these six years.
Focusing on the Linux Kernel and the Intermediate Model As research was conducted for this paper, it quickly became apparent that not all of the components within a given Linux distribution needed to be treated equally. Some projects, by their very nature and complexity, need different consideration under the COCOMO model. Nowhere was this more apparent than in Wheeler’s own follow-up to his 2002 article, where he highlighted the need for a different estimate model for the Linux kernel itself.12 In this new article, Wheeler maintained that because the Linux kernel code is typically more complex than an “average” application—among other things—it requires an analysis that goes beyond the basic COCOMO model. A user space application like Mozilla, for instance, is much easier to code line by line since it’s abstracted at a much higher level and has to handle far less tasks. A modern and enterprise-class operating system kernel is asked to do a great number of extremely complex things, all at once. Wheeler explains that instead of using the standard method used for analyzing all elements of a distribution, one should use the intermediate COCOMO model for counting just the kernel. The intermediate method has more variables to choose from, and therefore should be more accurate for analyzing a complex piece of software on its own. In the 2004 article, he details his estimates for these parameters, which have the net effect of adjusting the overall –effort factor value in SLOCCount from the default of 3 to a new value of 4.64607. At the same time, the –effort exponent value is also changed to 1.12, which is the value assigned to semi-detached software projects under the intermediate COCOMO software estimation model. With these new parameters in mind, a separate test was run on the stock Linux kernel that was included within Fedora 9, linux-2.6.25.i686:
Limitations and Advantages to this Study’s Approach There is no perfect way to estimate the value of something as complex and evolving as Linux. While this method we feel is the best approach, it likely over-counts some aspects of value, while under-estimates others. Here is a summary of those limitations:
How Has Linux Grown? This study also does not take into account the yearly development costs to incrementally update and grow the Linux code base. Based on these numbers (or anyone’s direct experience with a Linux distribution) it’s easy to see how the functionality included in a Linux distribution has exploded over the last six years. For instance, Fedora Linux 9 includes just over 204 million physical source lines of code (SLOC), compared to 30 million lines in Red Hat Linux 7.1 (issued in 2002), and over 17 million lines in version 6.2. So in just six years, the code base grew by 174 million lines of code. Using the COCOMO cost model, we estimate Fedora 9 to have required about 60,000 person-years of development time (as compared to 8,000 person-years for Red Hat 7.1 and 4,500 person-years for version 6.2). Thus, Fedora 9 represents a roughly 680% increase in size, a 750% increase in effort, and a 900% increase in traditional development costs over Red Hat Linux 7.1. This is due to an increased number of mature and maturing open source/ free software programs available worldwide. Such growth shows that Linux has a great deal of momentum: the continual addition of open source packages strengthens the applications set available to Linux users and in turn makes Linux that much more attractive as a computing platform. The modular nature of Linux (in its composition of a distribution) is also apparent by scanning the enclosed list of top ten packages included in a distribution. Given the time and interest, it would be interesting, for instance, to analyze the top components of an embedded Linux distribution and the development costs associated with that segment of computing. The impact in Linux innovation is not just measured in added lines. As the recent “Who Writes the Linux Kernel” paper states: “Over these releases, the kernel team has a very constant growth rate of about 10% per year, a very impressive number given the size of the code tree. But the kernel is not just growing. With every change that is made to the kernel source tree, lines are added, modified, and deleted in order to accomplish the needed changes.”14 While the Linux kernel probably has a higher change rate than most other components of Linux, our numbers reflect that as a whole Linux is growing and changing every year at a steady clip. Since the Linux kernel is just one small (but very important) component of a Linux distribution, the iterative development costs poured into Linux per year are substantial but not examined in this study.
Conclusions So do we now know what Linux is “worth?” While it may not be a question capable of being answered completely, some things are very clear: The true value of Linux springs from the ability to re-use it and the tremendous flexibility that it creates. Just imagine a computing world where Linus Torvalds didn’t allow (in fact force) users of Linux to allow others to re-use their contributions. Would there be a Google if they didn’t have the free use of Linux and the ability to modify it to suit their needs? Could there be the expanding new category of sub-$350 ultra mobile PCs without the free use of a $10.8 billion piece of software? Would Amazon be able to build its new line of Kindle reading devices without a free piece of $1.4 billion R&D to power it? More than just money, the software in a Linux distribution represents time. The economics in each of these examples would not have been possible had these companies been forced to pay per-device or per-server license fees to any one company or had to devote the thousands of person-years of development time to create this software. What can we learn from this study? The substantial development costs represented in a community-based Linux distribution reflect its increasing value and importance in the world of computing. The companies and individuals who work on Linux-related projects and build this value profit by sharing the development burden with their peers (and sometimes competitors.) Increasingly it’s becoming clear that shouldering this research and development burden individually, as Microsoft has done, is an expensive approach to building software. While monopoly position in the past has allowed them to fund this massive development, we believe that in the future competition from collaborative forces will make such an isolated position untenable. As we can see from this study and through the explosion of Linux throughout all areas of computing, collaborative development creates enormous economic value. Companies such as IBM, Intel, HP, Fujitsu, NEC, Hitachi, Google, Novell, Oracle, Red Hat, and many others have all participated and profited in the tremendous ecosystem created by this open model of software development. But make no mistake: while companies are participating, individuals are just as important in the expansion of this software and its value. That’s how it all began after all. The data in this paper was generated by SLOCCount, Copyright (C) 2001-2004 David A. Wheeler SLOCCount is Open Source Software/Free Software, licensed under the GNU GPL. Acknowledgements We would like to thank the following people for reviewing and providing feedback on this paper: James Bottomley, Jon Corbet, and David A. Wheeler.
Appendix For Fedora 9, the installation CD did not contain the source code, so the code was downloaded from the mirrors.kernel.org web site. Because determining what files to analyze would have entered another level of subjectivity into the process, it was decided that all 5547 available open source packages (src.rpms) would be downloaded and installed on the test machine. After downloading the src.rpms from the FTP server, it was then time to install them. Even though none of the rpms would be installed as binary RPMs (thus potentially being actually installed as applications on the test machine), a procedure was put in place to allow the source RPMs to be built and prepped without root access, modifying a procedure found on an archived Red Hat howto page15. The source code was downloaded to the test user’s home directory. Using gedit, a script was then created and saved as .rpmmacros in the same home directory. A destination directory set was created and then the src.rpm files were installed using this command: After a good amount of time, all 5547 packages were installed, with the specification files (.spec) residing in the rpm/SPECS directory and the source files and graphics populating the rpm/SOURCES directory. At this stage, the src.rpm files would have to built and prepped, which would align all of the applications’ source code into their own directories in the rpm/BUILD directory. To do this, the following command was used: Once this command was run, all of the packages were installed properly within the BUILD directory. The actual counting could then begin. Because a distribution is not a single software project, it should not be counted as such. SLOCCount provides a parameter to compensate for this: –multiproject. For the Fedora 9 report, the command used was: For further examination, following are the top 10 packages from the Fedora 9 source code:
|
- Dent Introduces Industry’s First End-to-End Networking Stack Designed for the Modern Distributed Enterprise Edge and Powered by Linux - 2020-12-17
- Open Mainframe Project Welcomes New Project Tessia, HCL Technologies and Red Hat to its Ecosystem - 2020-12-17
- New Open Source Contributor Report from Linux Foundation and Harvard Identifies Motivations and Opportunities for Improving Software Security - 2020-12-08
