Supercomputing Freakonomics – Finding Meaning Beyond the Headlines

Twice a year, the Top500 Project publishes its list of the fastest supercomputers in the world. In the last announcement, we continue to see Linux dominating the list. This is nothing new since Linux has been dominating since the mid-2000s. In fact, Linux share in supercomputing looks a lot like Microsoft’s historical share of the desktop market. I thought it would be interesting to take a step back and look at the performance capability of these computers as a whole and also how the rise of Linux is mirroring the geographical expansion of supercomputers.

Everybody tends to watch the number of Linux systems on the top500, but there’s a fascinating story being told by the Rmax performance numbers (Rmax is the maximum performance of a computer (measured in Gflop/s) achieved in the HPL benchmark. In many ways, this is a much more enlightening statistic, because it shows us the overall nature of performance on this list, instead of just focusing on individual computers. (This time around, five Linux systems were actually bumped off the bottom of the list, even though Linux’s *total* computing power grew by 38%.)

Linux dominance in the overall compute power of this list isn’t surprising since Linux is used in every one of the top ten computers, and especially when you consider that there is a full order of magnitude difference in performance between the first machine and the twelfth machine, which only increases as you move down the list. The first non-Linux system shows up at number 40.

Supercomputing power was on the rise well before Linux arrived, but when you look historically, it was Linux-powered machines that really caused the big ramp in the mid 2000s. In fact, when you graph the historical Rmax results of the top500 by OS, you can see that not only has supercomputing gone almost entirely to Linux, it’s also been the only OS driving the exponentially rising curve since 2005.

Next let’s look at where this is happening. This time Fujitsu of Japan tops the list. We have also seen players from China and Europe entering the fray. What’s really interesting is when you look at how this is distributed worldwide, and the role that Linux (including Linux machines that are classified as “Mixed,” like BlueGene) plays in making this happen.

It’s not surprising that the list has become very geographically diverse over time. What’s interesting is that this, too, is being driven almost entirely by Linux. In the graph below, all of the colored segments reflect the computing power deployed on “Linux” and “Mixed” machines in countries around the world. Blue is the US, white is Japan, red is China, orange is France, yellow is Germany, and so on. The dark segment on the bottom is all of the computing power deployed worldwide on platforms _other_ than Linux. Notice anything? (Here’s a hint, look at what OS is enabling this national diversity in supercomputing.)

Last, the good news here is that the there is more and more raw computing power being made available on a global basis thanks to Linux – and a lot of this innovation is making its way back into the kernel. As more countries start to use smart grid technology or seek to forecast the effects of global warming there is one common thread – the need for more computing power is endless. Just like what we’ve come to understand about Watson, (other than embarrassing humans at Jeopardy!), this technology will be used in smaller systems as we address one of the more pressing business issues of today – big data.

Once again these numbers are great for Linux. But more than numbers,, it is Linux’s ability to provide access to source code for anyone, to be optimized and have those optimizations returned for the common projects for ever increasing innovation that has created an unbreakable virtuous cycle in computing. I would also like to take this opportunity to congratulate our platinum member Fujitsu, who has done impressive research and development on Linux in super computers and for the enterprise, and who in this announcement has taken the number one position.