Kubernetes is everywhere.
82% of organizations using containers now run Kubernetes in production. (Source: CNCF, “Annual Survey 2023,” cncf.io/reports/cncf-annual-survey-2023) Kubernetes, Docker, and Terraform all written in Go are the infrastructure stack of the cloud-native era. (Source: Go Programming Language, golang.org; each project’s source repository on github.com/kubernetes, github.com/moby/moby, github.com/hashicorp/terraform)
Understanding how Kubernetes works has gone from a specialist skill to a professional baseline for anyone working in modern software, data, or cloud infrastructure. But Kubernetes has a well-earned reputation for a steep learning curve. The vocabulary alone can be disorienting. Nodes, clusters, pods, control planes, kubelets, schedulers, etcd these terms are used constantly and not always defined clearly.
This guide explains Kubernetes from the ground up. It covers what Kubernetes is, what problem it solves, and specifically what nodes and clusters are the two foundational concepts you need to understand before everything else starts making sense.
What Are Kubernetes?
Kubernetes (often abbreviated K8s) is an open-source container orchestration platform. Its job is to manage containerized applications across a cluster of machines.
A containerized application is packaged in a container typically Docker that includes the application code, its dependencies, and a runtime environment. Containers run consistently across different infrastructure. The problem Kubernetes solves is not running a single container that is straightforward. The problem is running hundreds or thousands of containers, across many machines, reliably and at scale.
Without an orchestrator, you would need to manually track which containers are running on which machines, restart containers that crash, route traffic to the right containers, scale up when load increases, and scale down to save cost when load decreases. All of this would require constant manual intervention and is error-prone at scale.
Kubernetes automates all of it. You declare the desired state “I want three instances of this service running at all times” and Kubernetes continuously works to make reality match that declaration. If a container crashes, Kubernetes restarts it. If a machine fails, Kubernetes reschedules its workloads onto healthy machines.
This is called the reconciliation loop: Kubernetes controllers continuously compare desired state to actual state and take corrective action when they differ.
What Is a Kubernetes Cluster?
A Kubernetes cluster is the aggregate of all the computing resources that Kubernetes manages the group of machines that collectively run your applications. Think of a cluster as a factory. The factory does not care which individual machine on the floor is producing a given component. It cares that the component is produced, to the right specification, within the required time.
The factory manager allocates work to machines and replaces broken equipment without the end customer ever knowing which specific machine was involved.
Kubernetes does the same for your applications. You do not specify “run this on server 3.” You specify “run three instances of this application.” Kubernetes decides which machines to use, and if one machine fails, it moves the workloads to others automatically. This abstraction is the primary value of Kubernetes. Your application code no longer needs to care about which physical or virtual machine it runs on, it just needs to run.
What a cluster is made of?
Every Kubernetes cluster consists of two types of components: the control plane and worker nodes.
The control plane is the management layer. It makes decisions about the cluster scheduling workloads, detecting failures, maintaining desired state but does not run application workloads itself. Worker nodes are the machines where application containers actually run. They receive instructions from the control plane and execute them.
In production, the control plane typically runs on dedicated machines separate from the worker nodes. This separation prevents application workloads from starving the control plane of CPU and memory during traffic spikes which would cause the cluster’s management layer to become unresponsive exactly when it needs to be most reliable.
What Is a Kubernetes Node?
A node is a machine in a Kubernetes cluster that runs containerized workloads.
Nodes can be physical servers or virtual machines. They can run in a data centre, in a public cloud (AWS EC2 instances, Google Compute Engine VMs, Azure virtual machines), or on-premises. To Kubernetes, the distinction does not matter if all nodes are managed through the same interface.
Each node is responsible for running the containers that the control plane schedules onto it, reporting its health and available resources back to the control plane, and maintaining the network rules that allow containers to communicate.
Worker nodes
Worker nodes are where your applications run. Each worker node runs three key Kubernetes components.
The kubelet is the primary agent on every worker node. It receives instructions from the control plane specifically, a description of which pods should be running on this node and makes sure those pods are running. If a container crashes, the kubelet detects the discrepancy and restarts it. The kubelet continuously reports node health and pod status back to the control plane.
The container runtime is the software that actually runs containers. For years this was Docker. Modern Kubernetes clusters typically use containers or CRI-O lighter-weight runtimes focused on the container execution role without Docker’s broader tooling.
Kube-proxy manages network rules on each node. It ensures that network traffic destined for a service is correctly routed to the pods that implement that service. It is what makes Kubernetes services the stable network endpoints that abstract over individual pods work correctly.
Control plane nodes
The control plane was historically called the “master node,” but this term was deprecated in Kubernetes 1.20. The current terminology is control plane node. (Source: Kubernetes, “Kubernetes 1.20 Release Notes,” kubernetes.io/blog/2020/12/08/kubernetes-1-20-release-announcement)
The control plane node runs four core components.
The API Server (kube-apiserver) is the single entry point for all cluster communication. Every kubectl command you run, every CI/CD pipeline action, every internal cluster communication goes through the API Server. It validates requests, authenticates users, and writes approved state changes to ETCD.
ETCD is a distributed key-value store and the single source of truth for cluster state. It stores the desired state of every object in the cluster pod specifications, service configurations, secrets, configmaps, and node registrations. If ETCD cd is lost without a backup, the cluster state is lost. Pods that are already running continue until they need rescheduling, at which point the control plane cannot recover its state.
The Scheduler (kube-scheduler) watches for newly created pods with no assigned node and selects the best node to run them on. Scheduling decisions take into account available CPU and memory on each node, affinity and anti-affinity rules (run this pod near that pod, or avoid running these pods on the same node), taints and tolerations (mark certain nodes as reserved for specific workloads), and resource limits and requests.
The Controller Manager (kube-controller-manager) runs background reconciliation loops one loop per Kubernetes resource type. The ReplicaSet controller ensures the correct number of pod replicas are running. The Node controller monitors node health and marks unresponsive nodes as NotReady. The Job controller manages batch workloads to completion. These loops are what make Kubernetes self-healing without human intervention.
Kubernetes Core Components: Quick Reference
| Component | Where It Runs | What It Does | What Breaks Without It |
| API Server | Control plane | Single entry point for all cluster communication; validates and stores state | No cluster operations — cannot deploy, scale, or modify anything |
| etcd | Control plane | Distributed key-value store; single source of truth for all cluster state | No state recovery after failure; pods continue until rescheduled, then lost |
| Scheduler | Control plane | Assigns pods to nodes based on resource availability and constraints | New pods remain unscheduled; existing pods unaffected until they need moving |
| Controller Manager | Control plane | Runs reconciliation loops for pods, nodes, jobs, and other resources | Self-healing stops; crashed pods not restarted; replica counts not maintained |
| kubelet | Worker node | Primary node agent; receives pod specs and ensures containers are running | Pods on this node cannot be managed; containers not started or restarted |
| kube-proxy | Worker node | Manages network rules for routing service traffic to correct pods | Service networking breaks; traffic cannot reach pods correctly |
| Container runtime | Worker node | Actually runs containers (containerd, CRI-O) | No containers can start on this node |
What Is a Pod?
A pod is the smallest deployable unit in Kubernetes. It is a wrapper around one or more containers that share the same network namespace and storage.
Containers within the same pod can communicate with each other via localhost. They share a network address; they are always scheduled and run together on the same node.
Most pods contain a single container. Multi-container pods are used for sidecar patterns where a helper container (for logging, metrics collection, or proxying) runs alongside the main application container.
Pods are ephemeral by design. Kubernetes does not try to repair a failing pod; it destroys it and creates a replacement. This means you should never rely on a pod persisting beyond its current lifetime. Persistent state (databases, file storage) must live outside pods in persistent volumes.
Because individual pods are disposable, you typically do not manage them directly. You use higher-level objects Deployments, StatefulSets, DaemonSets that manage groups of pods and handle creation, deletion, and replacement automatically.
Node vs Cluster vs Pod: The Relationship
These three concepts are nested.
- A cluster is the entire Kubernetes environment, all the machines, all the networking, the control plane, and all the workloads. It is the highest level of abstraction.
- A node is an individual machine within the cluster, a worker node that runs application workloads, or a control plane node that manages the cluster.
- A pod is the unit of work running on a node, a running container (or group of containers) that makes up part of your application.
To use the factory analogy: the cluster is the entire factory, the node is an individual machine on the factory floor, and the pod is the product being manufactured on that machine at a given moment.
How Kubernetes Decides Where to Run Things
When you submit a new deployment to Kubernetes, the sequence is as follows.
You send a manifest (a YAML file describing the desired state) to the API Server via kubectl or a CI/CD pipeline.
The API Server validates the manifest and stores the desired state in ETCD. The Scheduler detects the new pods with no assigned node. It evaluates all available worker nodes, checking available resources (CPU, memory), affinity rules, taints, and any scheduling constraints specified in the manifest. It assigns each pod to the best available node.
The kubelet on the selected node detects that a new pod has been assigned to it. It instructs the container runtime to pull the required images and start the containers.
The containers start running. kube-proxy updates network rules so the service can route traffic to the new pods.
The Controller Manager monitors the running pods. If any crash or are evicted, it creates replacement pods, and the scheduling cycle repeats for the replacements.
Managed Kubernetes vs Self-Managed
Running Kubernetes yourself, provisioning the nodes, configuring the control plane, managing ETCD backups, and upgrading the cluster is operationally complex. For most organizations, it is not the right choice.
All major cloud providers offer managed Kubernetes services that handle the control plane entirely. You provision worker nodes and deploy workloads. The cloud provider manages the API Server, ETCD , scheduler, and controller manager and takes responsibility for high availability and upgrades.
| Platform | Managed Service | What the Provider Manages |
| Amazon Web Services | Amazon EKS (Elastic Kubernetes Service) | Control plane, etcd, API server HA; you manage worker node groups |
| Google Cloud | Google GKE (Google Kubernetes Engine) | Control plane including auto-upgrade; Autopilot mode manages nodes too |
| Microsoft Azure | Azure AKS (Azure Kubernetes Service) | Control plane at no extra cost; you manage node pools |
| Self-managed | kubeadm, kops, Rancher, etc. | Everything — control plane, etcd, upgrades, backups, HA configuration |
The choice between managed and self-managed Kubernetes is mostly a question of what your team has bandwidth to operate. For teams whose core competency is not Kubernetes infrastructure, managed services reduce operational burden significantly and allow focus on application deployment rather than cluster management.
Why Kubernetes Matters for Data Teams
Kubernetes is not just a software deployment platform. In 2026, it will become the default runtime for data engineering infrastructure.
Apache Spark and Flink run natively on Kubernetes. Airflow orchestrates DAGs on Kubernetes pods. dbt projects run in containers scheduled on Kubernetes. ML training pipelines on PyTorch and TensorFlow run on GPU-enabled Kubernetes nodes. Data platforms like Databricks and Starburst have Kubernetes-native architectures.
For data teams building pipelines, running batch jobs, or deploying ML models, Kubernetes is the infrastructure layer that workloads run on regardless of whether the team manages it directly or works with a managed service. Understanding nodes, clusters, pods, and resource allocation is increasingly prerequisite knowledge for data engineers and ML engineers.
The architectural decisions around Kubernetes cluster sizing, node selection, resource requests and limits, and namespace organization directly affect data pipeline reliability, cost, and performance. Teams that understand the platform make better engineering decisions than those who treat it as a black box.
Final Thoughts
Kubernetes nodes and clusters are not advanced concepts, they are the foundational vocabulary for everything else in the Kubernetes ecosystem.
The cluster is the entire managed environment. The control plane makes decisions. Worker nodes execute them. Pods are what runs on the nodes. Understand these relationships and the rest of the architecture deployments, services, ingress, storage, namespaces follows logically.
For data platform teams building on cloud infrastructure whether that is running data pipelines on Kubernetes, managing compute for ML workloads, or building internal data tooling Data Pilot’s data engineering consulting helps teams make the infrastructure and architecture decisions that determine whether data platforms run reliably at scale.