Kubernetes Rolling Updates

Change a Deployment's pod template and Kubernetes replaces its Pods gradually. Starting new ones, waiting for them to become Ready, then removing old ones, so the application stays available throughout.

That is the theory. Whether it is actually zero-downtime depends on two things people routinely skip: a readiness probe and graceful shutdown.

What triggers one

Any change to spec.template creates a new ReplicaSet and starts a rollout:

kubectl set image deployment/payments-api api=ghcr.io/acme/payments-api:1.5.0
kubectl apply -f deployment.yaml            # if the template changed
kubectl rollout restart deployment/payments-api

Changes outside the template do not. Scaling replicas adjusts the current ReplicaSet in place and creates no revision.

kubectl rollout restart is worth knowing: it stamps an annotation on the template, which counts as a change and cycles every Pod through a normal rolling update. That is the correct way to pick up a modified ConfigMap or Secret consumed as environment variables, since those never refresh in a running process.

Watching it

kubectl rollout status deployment/payments-api

Waiting for deployment "payments-api" rollout to finish: 1 out of 3 new replicas have been updated...
Waiting for deployment "payments-api" rollout to finish: 2 of 3 updated replicas are available...
deployment "payments-api" successfully rolled out

kubectl get rs -l app=payments-api

NAME                      DESIRED   CURRENT   READY   AGE
payments-api-6c8d5f7b9    3         3         3       40s
payments-api-7d4f9c8b5    0         0         0       2d

The old ReplicaSet is kept at zero. That retained object is your rollback.

maxSurge and maxUnavailable

spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0

Both accept an integer or a percentage, and both default to 25%.

maxSurge: 1, maxUnavailable: 0 is the safe default: never fewer healthy Pods than you asked for, at the cost of one extra Pod's resources during the rollout. The trade is that the cluster must have room for that extra Pod, on a full cluster the rollout stalls with a Pending Pod and cannot progress.

The reverse, maxSurge: 0, maxUnavailable: 1, uses no extra capacity but runs one Pod short throughout. Reasonable on a tightly constrained cluster, and only if your remaining replicas can absorb the load.

At larger replica counts, percentages are more useful: maxSurge: 25%, maxUnavailable: 10% on 40 replicas rolls in sensible batches rather than one at a time.

The readiness probe is what makes it safe

During a rollout Kubernetes waits for each new Pod to become Ready before terminating an old one. Readiness is also what gates Service endpoints.

Without a readiness probe, Kubernetes treats a started process as ready. So it adds a Pod still loading configuration to the Service, and simultaneously terminates the next old Pod. The rollout reports success; users see 502s.

readinessProbe:
  httpGet:
    path: /healthz
    port: 8080
  initialDelaySeconds: 5
  periodSeconds: 5

If you set one thing on a Deployment, set this. maxUnavailable: 0 without a readiness probe gives you the illusion of safety, not safety.

minReadySeconds adds a further hold, a Pod must stay ready for that long before counting as available, which catches applications that pass their first probe and then fall over.

Graceful shutdown is the other half

When a Pod is terminated, two things happen concurrently, and the race between them is where dropped requests come from:

1. The Pod is removed from Service endpoints, which propagates to every node's kube-proxy.
2. The container receives SIGTERM.

Endpoint propagation is not instant. A container that exits immediately on SIGTERM will still be receiving traffic when it dies.

Two fixes, used together:

spec:
  terminationGracePeriodSeconds: 45
  containers:
    - name: api
      lifecycle:
        preStop:
          exec:
            command: ["sleep", "5"]

The preStop sleep delays SIGTERM long enough for endpoint removal to propagate. Meanwhile the application should handle SIGTERM by refusing new connections and finishing in-flight requests. After terminationGracePeriodSeconds, 30 by default, the container is SIGKILLed regardless, so the grace period must exceed your longest normal request.

Recreate, when you cannot run two versions

spec:
  strategy:
    type: Recreate

Terminates everything, then starts the new version. Downtime by design.

It exists for applications that genuinely cannot run two versions simultaneously: something holding an exclusive lock, a singleton worker, or a release with an incompatible schema migration. Do not choose it to make deploys simpler.

When a rollout stalls

error: deployment "payments-api" exceeded its progress deadline

progressDeadlineSeconds defaults to 600, so a stuck rollout reports failure after ten minutes. Critically it does not roll back. It stops, and the old Pods keep serving. A failed deploy is usually not an outage.

Diagnose the new Pods, not the Deployment:

kubectl get pods -l app=payments-api
kubectl describe pod <the-new-one>

To stop a bad rollout without reverting:

kubectl rollout pause deployment/payments-api
kubectl rollout resume deployment/payments-api

Pausing also lets you batch several edits into one rollout rather than triggering one per change.

Quick reference

---

Key takeaways

• Only pod template changes trigger a rollout. Scaling does not.
• maxSurge: 1, maxUnavailable: 0 keeps full capacity. Provided the cluster has room for one more Pod.
• A readiness probe is what makes a rolling update safe. Without one, "started" is taken to mean "ready".
• Graceful shutdown is the other half. A preStop sleep plus SIGTERM handling prevents requests dropped at termination.
• terminationGracePeriodSeconds must exceed your longest request, or in-flight work is killed.
• A stalled rollout stops and does not revert, so old Pods keep serving while you investigate.
• kubectl rollout restart is how you pick up a changed ConfigMap consumed as environment variables.

---

Check your understanding

---

Frequently Asked Questions

How do rolling updates work in Kubernetes?

Changing a Deployment's pod template creates a new ReplicaSet. The Deployment controller then scales the new ReplicaSet up and the old one down in steps, bounded by maxSurge and maxUnavailable, waiting for each new Pod to become Ready before removing an old one. The old ReplicaSet is retained at zero replicas so a rollback is just a matter of scaling it back up. Only template changes trigger this. Adjusting the replica count alone does not.

What is the difference between maxSurge and maxUnavailable?

maxSurge is how many Pods may exist above the desired replica count during the update, and maxUnavailable is how many of the desired Pods may be unavailable at once. Both default to 25% and accept integers or percentages. maxSurge: 1, maxUnavailable: 0 never drops below full capacity but requires room for one extra Pod; maxSurge: 0, maxUnavailable: 1 needs no extra capacity but runs one Pod short throughout.

How do I achieve genuinely zero-downtime deployments?

Four things together. Set maxUnavailable: 0 so capacity never dips. Define a readiness probe, without which Kubernetes treats a started process as ready and shifts traffic too early. Add a preStop hook that sleeps a few seconds, so endpoint removal propagates before the container gets SIGTERM. And handle SIGTERM in the application by refusing new connections while finishing in-flight ones, with terminationGracePeriodSeconds set above your longest normal request.

Why did my rollout stop with "exceeded its progress deadline"?

The new Pods failed to become Ready within progressDeadlineSeconds, which defaults to 600. Kubernetes marks the Deployment failed and stops. It does not roll back automatically, so with maxUnavailable: 0 the old Pods are still serving and the application is up. Inspect the new Pods with kubectl describe pod: the usual causes are a failing readiness probe, an image that cannot be pulled, a crash at startup, or no cluster capacity for the surge Pod.

When should I use the Recreate strategy?

Only when the application genuinely cannot tolerate two versions running simultaneously. Something holding an exclusive lock, a singleton background worker, or a release with a schema migration that the old version cannot work against. Recreate terminates every Pod before starting any new ones, so it causes downtime by design. It should be a deliberate decision about the application's constraints, never a way to simplify a deployment.

How do I make Pods pick up a changed ConfigMap?

kubectl rollout restart deployment/<name>. It sets a timestamp annotation on the pod template, which counts as a template change and triggers a normal rolling update honouring your surge and unavailability settings, so there is no downtime. This is necessary because ConfigMap values consumed as environment variables are fixed when the process starts and can never update in place. Values mounted as volumes do refresh automatically, within about a minute, unless mounted with subPath, which never updates.