Opening Doors to Collaboration with Open Source Projects

One of the biggest benefits of open source is the ability to collaborate and partner with others on projects. Another is being able to package and share resources, something Michelle Noorali has done using Kubernetes. In a presentation called “Open Source Opening Doors,” Noorali, a senior software engineer at Microsoft, told an audience at the recent LC3 conference in China about her work on the Azure containers team building open source tools for Kubernetes and containers.

Her team needed a way to reliably scale several containerized applications and found Kubernetes to be a good solution and the open source community to be very welcoming, she said.

“In the process of deploying a lot of microservices to Kubernetes we found that we wanted some additional tooling to make it easier to share and configure applications to run in our cluster,’’ she explained. “You can deploy and scale your containerized apps by giving Kubernetes some declaration of what you want it to do in the form of a Kubernetes manifest.” However, in reality, she added, to deploy one app to a cluster you may have to write several Kubernetes manifests that utilize many resources hundreds of lines long.

Helm

Once an engineer has come up with a set of Kubernetes manifests that work for them, they may want to package them up and share them with their team. To do that at Microsoft, she said, they started a project in 2015 called Helm, a package manager system that lets someone define their Kubernetes manifest in a format they call Charts. Inside a Chart is a set of files that can be used to template a manifest, Noorali said. Chart makes it easy to share the manifest.

Noorali had a colleague come on stage and do a brief Helm installation demonstration. Once a user completes the installation, they can give it a name, hit enter and “it will spit out deployment and services that have been created, which makes it so easy to deploy apps on Kubernetes,’’ she said.

It was “the spirit of openness and collaboration in the Kubernetes community [that] really opened the door for us to work with people from other organizations to build the solution together,” she noted. “This is no longer a zero-sum game and we on Helm lived that by collaborating with others in the community who would normally be competitors.”

Growing Pains

Helm today has over 345 contributors to the project and over 4,500 people in its Slack channel, according to Noorali.

But with growth comes growing pains, she observed. As they were scaling the code base and their team, they were also addressing people in the open source community who were building tooling on Helm and having conversations about its future. “What proved to also be challenging was meeting the non-technical needs of the community,” including people who were asking for mailing lists and taking the time to go into Slack channels to address questions, she said.

Her team also spent time “doing a lot around hardening roles and responsibilities and decision-making processes” as Helm became its own ecosystem and community.

The team turned to the Cloud Native Computing Foundation (CNCF) for support and then applied to become a “CNCF top-level incubating project,’’ and was ultimately accepted. Noorali called the transition “a really natural and smooth process.”

Looking ahead, she said they want Helm to continue becoming a bigger project, while at the same time, keeping both it and Kubernetes vendor neutral. This is important, she emphasized, “because we didn’t want any large company to come in and make decisions and steer the project in different direction,” and also because “this lowers the barrier for people who want to contribute to our project.”

Noorali also spent time discussing how to avoid repeating steps when developing containerized apps. She suggested people build an application container by writing a Docker file and using a Docker builder. “You’ll want to push an image to a container registry and create and install Helms charts in clusters.” A lot of tools are needed to deploy the app in Kubernetes and it’s important to understand the different layers of abstraction, she said.

Draft

The Draft tool is for people “who really just care about building their application and developing and debugging it against Kubernetes rather than working on and understanding Kubernetes itself,’’ she noted. “We live in a world where we can’t run all our microservices on our laptop to replicate a production-like environment.”

Draft and Kubernetes are helpful for iterating a local application with a remote service, Noorali said. “Draft allows you to do things like remote container debugging, testing out a local service with remote services [and] packaging all your container-related organizational patterns and conventions in what we call a ‘Draft pack.’”

This is an example of an early stage project and it is where a lot of people today are in the cloud native space, she said. “You may have a tool in your company that you’d like to open source because you see some benefits in that. You may want other people to use an open source project too, but they don’t always have insight into what makes a successful open source project.” This is where the CNCF can be very helpful, she said.

Watch the complete presentation below: