OSv: The Open Source Cloud Operating System That is Not Linux

For most tech professionals, the words “open source operating system” naturally translate to Linux. And so it’s understandable that those same tech pros would be a bit confused by startup Cloudius Systems’ announcement in September of a new open source operating system for the cloud, OSv.

While it was built completely on Linux, OSv is not Linux. It is an operating system written in C++ and designed to optimize the performance of a single application in the cloud. (For more of the technical details, see The Register’s coverage of OSv’s September launch.)

“It’s not a Linux kernel with a new userspace. It isn’t a Linux distribution,” said Avi Kivity, a co-founder of Cloudius and former co-maintainer of KVM, the Linux kernel-based virtual machine. “It’s a completely new kernel that we wrote from scratch and that’s why it’s so exciting.”

But while OSv isn’t Linux; it also isn’t a Linux competitor, says Glauber Costa, a Cloudius software engineer and also a former KVM developer.

“Even though we’re trying to replace Linux as a guest operating system, we’re not actually competing with Linux,” he said. “We’re part of the Linux ecosystem.”

Cloudius Systems joined the Linux Foundation as a silver member earlier this year. We met up at LinuxCon Europe with Kivity and Costa to talk about OSv’s relation to Linux; where it fits into the cloud stack; why a new operating system for the cloud is needed; how OSv makes a system administrator’s job easier; how to contribute to the project; and more.

How did you get started?

Avi: We were doing KVM for quite a long time. For three years at Qumranet and four more years at Red Hat and while it’s an amazing project and with a great trajectory and uptake, we wanted to do something new. Dor (Laor, a co-founder of Cloudius) was my manager at Qumranet and at Red Hat and we tried to look for areas we could both use our expertise but also do something new and interesting. And because we were doing the lower layer, we decided to move one layer up the stack and instead of doing the hypervisor do a guest.

When did you begin?

December 2012.

So wow, you’ve gotten a release together that quickly?

Avi: Yes, we released at CloudOpen in September and we had it up and running before that so we did move very quickly. And we think that with the community that’s starting to build around this project it’s going to run even more quickly.

Glauber Costa: I used to work with Avi and Dor and the others on KVM as well. I’ve worked on Xen in the past. So I’ve been working with hypervisors for a while. But I left Red Hat a few years ago to chase containers and when I did that I was kind of unhappy with the whole hypervisor situation. Because you have the hypervisor running and then you have another full blown operating system running on top of that and what we were trying to do with containers was something lighter which changed the operating system to give it the ability to run multiple things.

When Avi came to discuss with me the idea of OSv what I saw was a better way to do containers because you get the hardware to help you through the vmx extensions and the processor. It was a very different approach to something that was bothering me already.

Is OSv a Linux distribution?

Avi: No, that’s the part that almost everyone gets wrong because when you say ‘operating system’ it’s basically a synonym for Linux. We’re doing something completely new. It’s not Linux. It’s not a Linux kernel with a new userspace. It isn’t a Linux distribution. It’s a completely new kernel that we wrote from scratch and that’s why it’s so exciting. And that’s why it has a great potential. It’s not encumbered by all the decisions that were taken in Linux for 20 years and previously in Unix for even longer. So it’s a new take on the whole thing.

The reason why it’s possible even to think of something like that today is because of the cloud. Before if you wanted to write an operating system you had a huge set of hardware to support from different vendors; different processors, different network cards, different storage technology. And that’s almost impossible. But in the cloud there’s just one or two different types of virtual hardware and once you do them you have access to millions of machines.

That’s half of the equation. The other half is compatibility with applications and that is why we’re supporting the Java virtual machine as the major runtime. The reason is any application written for the Java VM is automatically compatible with OSv.

So we get compatibility with the hardware by running on top of the hypervisor and compatibility with the applications by using the JVM. All that’s left for us is to be the best intermediary between those two layers and try to be as small and lean as possible.

What are the key functions of that operating system?

Avi: We still need to do scheduling and provide a networking stack and a storage stack. And memory management as well, even though it’s a lot simpler. We are doing a lot less than a traditional operating system which is why we were able to be up and running so fast and why we’re able to provide better performance. We need to do less so we can do it faster. In addition, because we’re free from the traditional APIs we can do new things like integrate with the JVM and provide new APIs to the Java virtual machine which can make it run faster, especially in problem areas like garbage collection.

Who is your target?

Avi: We are looking at the cloud because it’s an environment that’s very agile. People know how to handle change. Instead of the usual enterprise cycle that used to be where you’d change an application once a year and prepare months in advance for it, people change their applications once a day or twice a day.

If we can show people the value, they will change. Afterwards of course we’d like to see not only cloud users adopt OSV, but also enterprise. Plus it’s a really good buzz word.

Can OSv run on top of Linux? 

Avi: Yes, Linux acts as the hypervisor with KVM. So OSV runs it. If you have a cloud that is based on KVM then you have OSv running on Linux.

Glauber: If you take the hypervisor as the layer for granted, then by all means OSv is an operating system. But if you look at the whole stack and you’re running KVM, which is essentially Linux, OSv is basically a library that you attach your application to and you can boot directly on KVM. You’re booting that application and using KVM as a containing mechanism.

Avi: And OSv is developed on Linux. We compile it on Linux, write on Linux, debug it on Linux. We like to see it as part of the Linux ecosystem.

In my view it’s basically containers, but using KVM as the dividing factor and not implementing everything inside of Linux and software which is not really very easy to do.

It’s also Xen compatible?

Yes, it’s mostly for Amazon which is based on Xen. It’s by far the largest cloud. Most of the others are using KVM. We support all commerce, we want to grow the reach as much as possible.

What else should Linux sysadmins know about your operating system?

Glauber: Vacations. We are making the lives of sysadmins in general so much easier. In a traditional operating system you have to configure it and tune it to the actual thing that you’re doing. People try to automate that but it’s still a laborious process. Its error prone and you always have to have supervision. With OSv we’re running just one application so we don’t have configuration files. You don’t have to configure anything. You don’t have to tune anything. It’s just really being able to build this one single workload. For the sysadmin managing a guest operating system, his life is much easier now.

Avi: The traditional OS supports multiple users with multiple applications running on top. OSV doesn’t do any of that. It trusts the hypervisor to provide the multi-tenancy and it just runs a single one. We do a smaller job and that allows us to do it well and allows the administration of that system to be easeir as well because there’s simply less to administer.

What about app developers?

Avi: For applications, if it’s a JVM-based application – and that doesn’t just mean Java but applications based on JVM-compatible languages like Scala, Ruby or Javascript or a number of others can run without change on OSv. Non JVM applications are a little trickier. We don’t expect all applications to run unmodified. There will have to be some effort porting. There will probably be just a select number of applications that have a large benefit and are widely used, such as memcached, can be ported and we will do some of the porting ourselves and will work with the community to do those ports where it makes sense.

Why was this operating system necessary, or an opportunity that no one else has realized?

Avi: It became possible with the advent of the cloud. But you really see the pain for people that are managing large amounts of instances. Applications are scaled to more and more virtual machines and there is a need for something that is easily managed and gives good performance because you’re paying for every hour your application runs. If you’re running fewer instances you’re paying less money to a cloud provider. It saves effort and it saves money.

Glauber: I also see no theoretical difference between what is a hypervisor and what’s an operating system. So with KVM those barriers became even blurrier – now KVM is it is the hypervisor and the operating system. And if you think about the whole stack, you already have an operating system. So what you did before is you had multiple different versions of Linux running to get the whole cloud stack done and we really only need an operating system which is the hypervisor. 

How is this a business?

Avi: At the moment it is an open source project but we do plan to make money so we will offer a subscription service for customers who like support and updates on a more formal basis. But that’s in the future. We haven’t yet launched our commercial product.

Glauber: We’re here for developers at this stage. 

Even though we’re trying to replace Linux as a guest operating system, we’re not actually competing with Linux. We’re part of the Linux ecosystem. If you have KVM, especially, you always have Linux in there, you’re just shuffling around a bit the way you do things.

How did you build it?

Avi: We used the regular toolchain: GCC, Git, Make, openJDK – all the regular libraries that make up Linux. Some of them we used unchanged, some we modified and it’s all open source.

Glauber: We are technically using C++, which is unusual in the operating system and especially in Linux. I used to be a C++ hater – Avi always loved C++. Once he had this big plan to convert Qemu to C++, the project that provides the emulation for KVM. And I even had a planned assassination attempt on his life. Things are different now. This is one of the things that allowed us to be fast in terms of development, a lot of the things Linux as an operating system implements in terms of internal data structure we’re not really doing that. We have a C++ standard library providing them.

Avi: We’re not just doing something new but in new ways.

Can you find contributors who understand that and contribute?

Avi: We already have. It’s great to see the community picking up. We’ve already gone through this process once and it’s impressive to see how people come from all over the world.

What’s your next phase?

Avi: We’re building the system up to be able to show the performance and getting it ready to be run as production. Right now it works very well but I wouldn’t put all my mission critical servers on it just yet. We’re working on adding missing functionality, improving performance and most of all productizing it so it can be ready. Perhaps still in a limited way, but we’ll have a few applications that will be well tested. 

Which applications are you targeting first?

Avi: Right now we’re looking at Cassandra and memcached. We could do a few others, possibly Tomcat or similar.

How do developers get their hands on OSv?

Avi: All of our code is on Github or on our site, osv.io. We’re using the standard Linux mechanism for contributions so far. So it’s a very welcoming experience for people coming from Linux because we’re doing patches to the mailing list, so the whole process looks like Linux.

There’s an image available on Amazon if you’re not a developer and want to take it for a spin. And if you’re a developer you can clone the Git repository and follow a few steps on the read.me and type make and you’re all set. You can run it via KVM on your laptop and it’s a nice small system that’s easy to run.