Amazon Web Services Aims for More Open Source Involvement

In 2006, Amazon was an e-commerce site building out its own IT infrastructure in order to sell more books. Now, AWS and EC2 are well-known acronyms to system administrators and developers across the globe looking to the public cloud to build and deploy web-scale applications. But how exactly did a book seller become a large cloud vendor?

Amazon’s web services business was devised in order to cut data center costs – a feat accomplished largely through the use of Linux and open source software, said Chris Schlaeger, director of kernel and operating systems at Amazon Web Services in his keynote talk at LinuxCon and CloudOpen Europe today in Dusseldorf.

Founder Jeff Bezos “quickly realized that in order to be successful in the online business, he needed a sophisticated IT infrastructure,” Schlaeger said. But that required expensive proprietary infrastructure with enough capacity to handle peak holiday demand. Meanwhile, most of the time the machines were idle

By building their infrastructure with open source software and charging other sellers to use their unused infrastructure, Amazon could cover the up front cost of data center development.

“That was our first service model,” said Schlaeger, who is also managing director of Amazon Development Center in Germany.

It built a foundation of infrastructure and services including storage, security and access, followed by compute and networking and content delivery. Then, on top it built a range of platform services such as databases, analytics, app services, deployment and management services, and services for mobile devices.

“It allows customers to focus their attention on their business rather than become IT experts,” Schlaeger said. “This is a sophisticated view of clouds, because it’s not just foundation services. You need to offer more.”

Accelerated growth

By 2008 AWS offered 24 features and services, roughly doubling the pace of development every year since. Last year the company offered 280 different services. And AWS is on track for the same accelerated development pattern again this year, Schlaeger said.

At the same time AWS has cut prices 45 times – the result of what Schlaeger calls, “The AWS Innovation Flywheel.” New features and services lead to more AWS users, which requires more infrastructure which leads to greater economies of scale and greater innovation, which leads to reduced costs – a continuous cycle.

Behind all of these new services and infrastructure innovations lies Amazon Linux, the company’s custom operating system. Used internally and also available to AWS customers, it provides a stable and secure base on which customers can build and run their own applications and services.

“In order to innovate, the ability to modify every parameter of your system is key and Linux and open source is a good basis for that,” Schlaeger said.

Amazon has created a custom operating system that requires minimal overhead and has key security features such as automatic updates of critical packages at launch, no root login by default, regular updates, and new AMIs (Amazon Machine Images) twice a year.

As new technologies come along, Amazon can incorporate them to the benefit of their own services as well as their customers. The speed with which AWS has incorporated Docker, a relatively new and fast-growing project, is indicative of Amazon’s increased participation in the open source community over the past year. Docker has already been incorporated with Amazon EC2, AWS Elastic Beanstalk and AWS OpsWorks, Schlaeger said.

“We’re really serious about getting engaged with the open source community and gaining more visibility,” Schlaeger said.