Chef Offers a Recipe for the Open Source Cloud

Editor’s note: I spoke with Opscode CTO Chris Brown last week in advance of his keynote panel, “IaaS vs. PaaS: Composable primitives for building in the cloud,” at CloudOpen, Aug. 29-31 in San Diego. Opscode is also offering a Chef 101 Workshop at CloudOpen on Tuesday, Aug. 28. This profile is part of a series featuring Leaders of the Open Cloud that will run every week until the conference.

As one of the original architects of Amazon’s EC2, Opscode CTO Chris Brown witnessed firsthand what happens when you make ubiquitous and nearly infinite computing power available to engineers who were used to working with a handful of machines. In short, you become very, very popular.

But all of that additional computing capacity created a new problem for IT managers. 

“When you have that capacity at hand you have no idea how to manage it,” said Brown. “The next thing you need is automation to dynamically mange that.”

Opscode saw an opportunity to solve this problem and built the open source Chef framework for automating application development on the cloud. In the process, they’ve also come up with a template for how the open source cloud could operate.

“At the beginning we all wanted to treat the cloud as an easier way to get the compute units that looked like the old thing we used to get buying a physical machine,” Brown said. “But that’s not actually true. Designing the cloud and for the cloud are different than what is now legacy.”

Developing for the Cloud

Born of the same mold as Puppet, Chef is part of the orchestration layer of platform-agnostic software aimed at allowing developers to create a reproducible blueprint for server configuration and automation. But Chef goes beyond simply managing the machines and aims to create a new development model that’s fluid, a concept known as “infrastructure as code.”

The fluid approach puts Opscode at the cutting edge of DevOps. Developers and system administrators work together from a common cookbook or recipe – Chef’s blueprint for what the application requires from the system and vice versa. It’s an approach that requires companies accustomed to operating in a traditional IT environment to pitch those old notions out the window.

“If you treat compute as a throwaway resource it removes some stress and allows you to radically change the way you design,” he said. “The legacy of the server huggers, those days are gone. It’s very easy to correct your mistakes in a dynamic environment.

“It means a much more hopefully streamlined and visible way to do app deployment and version tracking,” Brown said, “to understand the interdependencies of components and infrastructure.”

IaaS vs. PaaS, a False Debate?

Looking at the broader picture of cloud services, IaaS and PaaS architects could take a lesson from Chef in finding a common recipe for communicating capabilities between machines in a cloud or between clouds, Brown says. As it is, no two implementations allow you to define the details of security and networking the same way, he says.

“We almost need a declarative language for the cloud to describe how the pieces work together so you can build up with the pieces you want,” Brown said.

A language that allows for such interoperability would end the “artificial debate” happening now in the industry over whether IaaS or PaaS is the better use case in balancing cost and control, Brown said. Some say Infrastructure-as-a-Service is “too primitive,” Brown said, requiring specialists to build in the cloud and creating high developments costs. On the other side, Platform-as-a-Service is “too sealed” and developing on one platform can make it harder to change platforms down the line.

A common language would create a truly open cloud in which companies could “pick up that language and move from one cloud to another,” he said.

“I don’t know what that language ought to look like,” he said, “But it would be able to, in a programmatic fashion, make those demands of a cloud system and see how they perform.”

Brown will take a deep dive into this issue, along with Kyle MacDonald of Canonical and Peter Magnusson of Google App Engine in the keynote panel, “IaaS vs. PaaS: Composable primitives for building in the cloud,” at CloudOpen, Aug. 29-31 in San Diego.