Kubernetes Architecture

A Kubernetes cluster splits into two halves. The control plane decides what should happen. The nodes make it happen.

Knowing which component does what is not trivia. It is what turns "the cluster is broken" into a specific thing to check.

Control plane

kube-apiserver

The only door into the cluster. Every component, kubectl, controllers, the scheduler, the kubelets on every node, talks exclusively to the API server, and none of them talk to each other or to etcd directly.

It authenticates the caller, authorises the request against RBAC, runs admission controllers that can mutate or reject the object, validates it, and writes it to etcd.

It is stateless, so it scales horizontally, and production clusters run several behind a load balancer.

When it fails: you lose all control. kubectl stops working, no deployments, no scaling, no self-healing. Running Pods keep running and keep serving traffic, because nodes do not need it to keep doing what they are already doing.

etcd

A distributed key-value store holding the entire cluster state. Every object, its spec and its status. Nothing else in Kubernetes is stateful. Lose etcd without a backup and you have lost the cluster, even with every node still healthy.

It uses Raft consensus, which is why you run an odd number of members. A three-member cluster tolerates one failure; five tolerates two. Four is worse than three. More nodes to fail, the same tolerance.

etcd is sensitive to disk latency in a way people underestimate. Slow disks produce leader elections, which look like random cluster-wide instability. Give it fast SSDs.

# Back it up. This is the one thing you cannot recreate.
ETCDCTL_API=3 etcdctl snapshot save /backup/etcd-$(date +%F).db \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key

On a managed cluster, EKS, GKE, AKS, the provider runs and backs up etcd, which is a large part of what you are paying for.

kube-scheduler

Watches for Pods with no node assigned and picks one. It does not start anything; it writes the chosen node onto the Pod, and that node's kubelet does the rest.

Two phases: filtering removes nodes that cannot run the Pod (not enough free CPU or memory, a taint with no matching toleration, an unsatisfiable node selector or affinity rule, no access to the required volume), then scoring ranks the survivors and the highest wins.

When it fails: new Pods sit in Pending forever. Everything already running is unaffected.

That is also the clue when a single Pod is stuck Pending. The scheduler filtered out every node, and kubectl describe pod reports exactly why:

Warning  FailedScheduling  0/5 nodes are available:
  2 Insufficient memory, 2 node(s) had untolerated taint, 1 node(s) had volume node affinity conflict.

kube-controller-manager

One binary running many controllers, each an independent loop: observe current state, compare to desired state, act on the difference.

When it fails: nothing self-heals. Deleted Pods are not replaced, failed nodes are not drained, deployments do not progress. Existing workloads keep serving.

cloud-controller-manager

Where present, this is the part that talks to the cloud provider: provisioning load balancers for LoadBalancer Services, attaching storage volumes, and updating node metadata such as region and zone.

When it fails: LoadBalancer Services stay <pending> and volumes do not attach.

Node components

kubelet

The agent on every node. It watches the API server for Pods assigned to it, tells the container runtime to start them, mounts their volumes, runs their probes, and reports status back.

It is the component that actually runs your containers, and it is deliberately independent: a kubelet that cannot reach the API server keeps its existing containers running. That is why a control plane outage does not stop traffic.

When it fails: the node stops reporting and is marked NotReady. After roughly five minutes the node controller starts evicting its Pods so they can be rescheduled elsewhere, but the containers on the failed node may still be running, which is why a partitioned node is a genuinely awkward failure.

kubectl get nodes
kubectl describe node <name>    # conditions: MemoryPressure, DiskPressure, Ready

kube-proxy

Implements Services. It watches Services and EndpointSlices and programs the node's iptables or IPVS rules so traffic to a Service's virtual IP is rewritten to a real Pod IP.

The key insight: there is no proxy process in the request path. kube-proxy writes kernel rules and gets out of the way. That is why Service load balancing is per-connection and cannot inspect HTTP.

When it fails: existing rules stay in place so current Services keep working, but new Services and changed endpoints are not programmed.

Container runtime

Actually runs the containers, containerd most commonly, or CRI-O. Kubernetes talks to it through the Container Runtime Interface.

Docker was removed as a supported runtime in Kubernetes 1.24. This changed nothing about images: Docker builds OCI images and containerd runs them.

What happens when you run kubectl apply

1. kubectl sends the manifest to the API server.
2. The API server authenticates, authorises via RBAC, runs admission controllers, validates, and writes to etcd.
3. The Deployment controller sees a new Deployment, creates a ReplicaSet.
4. The ReplicaSet controller sees it needs Pods, creates Pod objects with no node assigned.
5. The scheduler sees unscheduled Pods, filters and scores nodes, writes the chosen node onto each Pod.
6. The kubelet on that node sees a Pod assigned to it, asks the container runtime to pull the image and start the container, mounts volumes, and starts running probes.
7. The kubelet reports status back to the API server.
8. Once the Pod is Ready, the endpoints controller adds its IP to the EndpointSlice, and kube-proxy on every node programs rules so the Service reaches it.

Nothing in that chain is a direct call between components. Each one watches the API server and reacts. That is why Kubernetes is resilient to any single piece being briefly unavailable, and why "eventually consistent" describes its behaviour better than "immediate".

Control plane vs node, side by side

Control plane nodes normally carry a taint preventing workloads from being scheduled there. Removing it on a single-node dev cluster is standard; doing it in production means an application can starve the control plane.

Quick reference

---

Key takeaways

• The API server is the only entry point. Nothing talks to etcd or to another component directly.
• etcd is the only stateful component. No backup, no cluster. Back it up deliberately, and give it fast disks.
• The scheduler only assigns; the kubelet runs. A Pending Pod is a scheduling problem, and describe pod names it.
• Controllers are reconciliation loops, which is why self-healing stops when the controller manager does.
• kube-proxy programs kernel rules and leaves the data path, so Service balancing is per-connection.
• Kubelets keep running their containers without the control plane: a management outage is not a traffic outage.
• Run an odd number of etcd members. Four gives you no more tolerance than three.

---

Check your understanding

---

Frequently Asked Questions

What are the main components of Kubernetes architecture?

The control plane comprises the API server (the single entry point for every request), etcd (the key-value store holding all cluster state), the scheduler (assigns Pods to nodes), the controller manager (runs the reconciliation loops that provide self-healing), and on cloud clusters the cloud controller manager (provisions load balancers and volumes). Each worker node runs the kubelet (starts and monitors containers), kube-proxy (programs Service networking rules), and a container runtime such as containerd.

What happens if the Kubernetes control plane goes down?

Running Pods keep running and keep serving traffic. Kubelets deliberately continue managing the containers they already have when they cannot reach the API server, and kube-proxy's existing kernel rules keep routing Service traffic. What stops is change: no deployments, no scaling, no rescheduling of failed Pods, and kubectl does not work. It is a control outage rather than a service outage, though it becomes a service outage if something fails during it, since nothing will replace it.

Why does etcd need an odd number of members?

etcd uses Raft consensus, which requires a strict majority of members to agree before committing a write. Three members need two healthy and tolerate one failure; five need three and tolerate two. A four-member cluster still needs three healthy, so it tolerates exactly one failure, the same as three members, while giving you an extra machine that can fail. Even numbers add cost and risk without adding fault tolerance.

What is the difference between kubelet and kube-proxy?

The kubelet manages containers: it watches for Pods assigned to its node, instructs the container runtime to start them, mounts volumes, executes liveness and readiness probes, and reports status to the API server. kube-proxy manages Service networking: it watches Services and EndpointSlices and programs iptables or IPVS rules so traffic sent to a Service's virtual IP is rewritten to a real Pod IP. One runs your workloads, the other makes them reachable.

Does Kubernetes still use Docker?

Not as a runtime. Kubernetes removed the Docker shim in version 1.24 and uses containerd or CRI-O through the Container Runtime Interface. This has no effect on images: Docker produces OCI-compliant images and containerd runs them, so anything built with docker build works unchanged. The only practical difference is on the nodes themselves, where docker ps no longer shows Kubernetes containers, crictl ps does.

Why is my Pod stuck in Pending?

The scheduler filtered out every node and found nowhere to place it. kubectl describe pod <name> reports the exact reason in its events, usually something like 0/5 nodes are available: 2 Insufficient memory, 2 node(s) had untolerated taint. The frequent causes are resource requests larger than any node's free capacity, taints without matching tolerations, node selectors or affinity rules that match nothing, and a PersistentVolumeClaim that cannot bind or is in the wrong Availability Zone.