How Kubernetes is Built with Kat Cosgrove
How Kubernetes is Built with Kat CosgroveKat Cosgrove shares what Kubernetes actually does, how it’s structured and scaled, and why it’s become one of the most successful open-source projects in the world.
Stream the Latest EpisodeListen and watch now on YouTube, Spotify and Apple. See the episode transcript at the top of this page, and timestamps for the episode at the bottom. Brought to You By• WorkOS — The modern identity platform for B2B SaaS. • Modal — The cloud platform for building AI applications. • Cortex — Your Portal to Engineering Excellence. — In This EpisodeKubernetes is the second-largest open-source project in the world. What does it actually do—and why is it so widely adopted? In this episode of The Pragmatic Engineer, I’m joined by Kat Cosgrove, who has led several Kubernetes releases. Kat has been contributing to Kubernetes for several years, and originally got involved with the project through K3s (the lightweight Kubernetes distribution). In our conversation, we discuss how Kubernetes is structured, how it scales, and how the project is managed to avoid contributor burnout. We also go deep into:
 TakeawaysSome of the most interesting topics discussed in the conversation were these: 1. Kubernetes helps with managing large-scale backend applications. Google originally built an in-house tool to manage the tens of thousands (then hundreds of thousands and later millions) of machines: this internal tool is called Borg. The roots of Kubernetes come from Borg: but Google has since donated the project to the Cloud Native Computing Foundation (CNCF) – and today, Kubernetes is the second largest open source project, after Linux. We previously did a deepdive on How Linux is built, and touched on Google’s SRE roots in What is Reliability Engineering? 2. Kubernetes is a very well-structured and organized open source project. The structure of the project and all processes are well documented. The project has around 150-200 maintainers, has a few dozen SIGs (Special Interest Groups) and releases run on a 14-16 week cycle. 3. The “lead” and “shadow” concept is a clever one, utilized by Kubernetes. The Release Team within Kubernetes owns releases, and the Release Team has about 20-30 people participating in each release. About half of the participants in the Release Team are “shadows” who get to learn on the job how a release is done – and, hopefully, in a release or two, become leads themselves! Unlike most open source projects where getting a spot on the release team is based on long tenure and impactful contributions: the Kubernetes team recruits “shadows” people via an application process. Even those with no prior Kubernetes contributions are invited to join and participate. To get notified of applications opening, subscribe to the relevant Kubernetes mailing lists. This is a very friendly policy – and another reminder that a project can come up with its own policies: no need to copy existing ones. An interesting quote: why did Kubernetes “win”?From the episode, starting at 29:26
The Pragmatic Engineer deepdives relevant for this episodeTimestamps(00:00) Intro (02:02) An overview of Kubernetes and who it’s for (04:27) A quick glimpse at the architecture: Kubernetes components, pods, and cubelets (07:00) Containers vs. virtual machines (10:02) The origins of Kubernetes (12:30) Why Google built Borg, and why they made it an open source project (15:51) The benefits of open source projects (17:25) The size of Kubernetes (20:55) Cluster management solutions, including different Kubernetes services (21:48) Why people contribute to Kubernetes (25:47) The anti-burnout policies Kubernetes has in place (29:07) Why Kubernetes is so popular (33:34) Why documentation is a good place to get started contributing to an open-source project (35:15) The structure of the Kubernetes release team (40:55) How responsibilities shift as engineers grow into senior positions (44:37) Using a KEP to propose a new feature—and what’s next (48:20) Feature flags in Kubernetes (52:04) Why Kat thinks most GenAI tools are scams—and why Kubernetes blocks their use (55:04) The use cases Kat would like to have AI tools for (58:20) When to use Kubernetes (1:01:25) Getting started with Kubernetes (1:04:24) How contributing to an open source project is a good way to build your network (1:05:51) Rapid fire round ReferencesWhere to find Kat Cosgrove: • Bluesky: https://bsky.app/profile/kat.lol • LinkedIn: https://www.linkedin.com/in/katcosgrove/ Mentions during the episode: • Kubernetes: https://kubernetes.io/ • Docker: https://www.docker.com/ • Mesos: https://mesos.apache.org/ • Borg: https://en.wikipedia.org/wiki/Borg_(cluster_manager) • The Linux Foundation: https://www.linuxfoundation.org/ • Cloud Native Computing Foundation: https://www.cncf.io/ • Joe Beda on LinkedIn: https://www.linkedin.com/in/jbeda/ • Seven of Nine” https://en.wikipedia.org/wiki/Seven_of_Nine • What is Reliability Engineering?: https://newsletter.pragmaticengineer.com/p/reliability-engineering • Dave O’Conner on LinkedIn: https://www.linkedin.com/in/gerrowadat/ • Spotify for Backstage: https://backstage.spotify.com/ • Azure Kubernetes Service: https://azure.microsoft.com/en-us/products/kubernetes-service • Oracle Kubernetes Engine: https://www.oracle.com/cloud/cloud-native/kubernetes-engine/ • RedHat Openshift: https://www.redhat.com/en/technologies/cloud-computing/openshift/ • VMware Tanzu: https://www.vmware.com/products/app-platform/tanzu • 2347: Dependency: https://www.explainxkcd.com/wiki/index.php/2347:_Dependency • Inside Linear's Engineering Culture: https://newsletter.pragmaticengineer.com/p/linear • Linear: move fast with little process (with first engineering manager Sabin Roman): https://newsletter.pragmaticengineer.com/p/linear-move-fast-with-little-process • Linear: https://linear.app/ • PEP: https://peps.python.org/pep-0001/ • Google Kubernetes Engine: https://cloud.google.com/kubernetes-engine • How Linux is built with Greg Kroah-Hartman: https://newsletter.pragmaticengineer.com/p/how-linux-is-built-with-greg-kroah • KEPs: https://www.kubernetes.dev/resources/keps/ • The Philosophy of Software Design – with John Ousterhout: https://newsletter.pragmaticengineer.com/p/the-philosophy-of-software-design • Python: https://www.python.org/ • A Fire Upon the Deep: https://www.amazon.com/Fire-Upon-Deep-Zones-Thought/dp/0812515285 • Kubernetes on Slack: https://communityinviter.com/apps/kubernetes/community — Production and marketing by Pen Name. You’re on the free list for The Pragmatic Engineer. For the full experience, become a paying subscriber. Many readers expense this newsletter within their company’s training/learning/development budget. This post is public, so feel free to share and forward it. If you enjoyed this post, you might enjoy my book, The Software Engineer's Guidebook. Here is what Tanya Reilly, senior principal engineer and author of The Staff Engineer's Path said about it:
|
%26stroke%3Dnone%26strokeWidth%3D3.6){kind=small}
Comments
Post a Comment