Kubernetes Rollbacks

A rollback returns a Deployment to an earlier pod template. It is fast, it is one command, and it works because Kubernetes keeps the old ReplicaSets around at zero replicas.

It is also more limited than people assume during an incident, so it is worth knowing precisely what it does and does not restore.

Revision history

kubectl rollout history deployment/payments-api

deployment.apps/payments-api
REVISION  CHANGE-CAUSE
1         <none>
2         <none>
3         <none>

Every pod template change creates a revision, each backed by a retained ReplicaSet:

kubectl get rs -l app=payments-api

NAME                      DESIRED   CURRENT   READY   AGE
payments-api-6c8d5f7b9    3         3         3       4m
payments-api-7d4f9c8b5    0         0         0       2d
payments-api-5b9c7d446    0         0         0       9d

Those zero-replica ReplicaSets are the rollback mechanism. Rolling back scales one of them back up.

Inspect a revision before choosing it:

kubectl rollout history deployment/payments-api --revision=2

Pod Template:
  Labels:  app=payments-api
  Containers:
   api:
    Image:  ghcr.io/acme/payments-api:1.4.2

Record why, not just what

That <none> column is the single most common regret during an incident. A list of revision numbers with no descriptions tells you nothing about which one is safe.

CHANGE-CAUSE comes from an annotation you have to set:

kubectl annotate deployment/payments-api \
  kubernetes.io/change-cause="Deploy 1.5.0 - new pricing engine" --overwrite

Better, put it in the manifest so it ships with the change:

metadata:
  annotations:
    kubernetes.io/change-cause: "Deploy 1.5.0 - new pricing engine (PR #482)"

REVISION  CHANGE-CAUSE
1         Initial release 1.3.0
2         Deploy 1.4.2 - fix timeout handling (PR #455)
3         Deploy 1.5.0 - new pricing engine (PR #482)

Have CI set it automatically from the commit message or tag. It costs one line and it is what makes the history usable at 3am.

Rolling back

# To the immediately previous revision
kubectl rollout undo deployment/payments-api

# To a specific revision
kubectl rollout undo deployment/payments-api --to-revision=2

# Watch it complete
kubectl rollout status deployment/payments-api

A rollback is itself a rolling update, so it honours maxSurge and maxUnavailable and waits on readiness probes. It is not instantaneous and it is not a hard cutover.

It also creates a new revision rather than deleting history. Rolling back from 3 to 2 produces revision 4 containing revision 2's template:

REVISION  CHANGE-CAUSE
1         Initial release 1.3.0
3         Deploy 1.5.0 - new pricing engine (PR #482)
4         Deploy 1.4.2 - fix timeout handling (PR #455)

Revision 2 is renumbered as 4; the history stays complete and auditable, which matters for a post-incident review.

How far back you can go

spec:
  revisionHistoryLimit: 10      # the default

Only this many old ReplicaSets are kept. Beyond that the oldest are deleted and those revisions become unrollbackable.

The default of 10 is reasonable. Setting it to 0 disables rollback entirely, which occasionally appears in manifests written to reduce clutter and removes your fastest recovery option. A few extra zero-replica ReplicaSets cost nothing but an API object.

What a rollback does not fix

This is the part worth internalising before you need it.

Database migrations. If release 1.5.0 ran a migration, rolling back the Deployment leaves it applied. If that migration dropped a column 1.4.2 depends on, the rollback makes things worse. This is why expand-and-contract migrations exist: deploy schema changes that both versions tolerate, and only remove the old shape once the new version is proven.

Deleted data. A rollback restores objects, not contents. A removed PersistentVolumeClaim comes back as an empty claim.

Anything outside this Deployment. ConfigMaps, Secrets, CRDs, other services. A rollback reverts one Deployment's pod template and nothing else, which is a strong argument for versioning ConfigMap names so config changes ride the same rollout.

In-flight side effects. Messages published, emails sent, webhooks fired. Already gone.

Rollback or roll forward?

Rolling back is the right instinct when the previous version is known good and the failure is clear. It is fast and it reduces the blast radius while you investigate.

Roll forward instead when the new version has already changed persistent state in a way the old one cannot handle, when the previous version has a known problem of its own, or when the fix is genuinely small and well understood. A rollback that reintroduces a different bug is a bad trade.

If you are unsure, roll back. Restoring a known state is almost always better than debugging under pressure.

Preventing the need

# Fail fast rather than waiting ten minutes
kubectl patch deployment payments-api -p '{"spec":{"progressDeadlineSeconds":300}}'

# See what will change before applying
kubectl diff -f deployment.yaml

A correct readiness probe is the real protection: with maxUnavailable: 0, a version that never becomes Ready never displaces the old Pods, so the bad release stalls instead of taking traffic. The rollout fails and the application stays up, which is a much better outcome than a successful rollout of broken code.

Quick reference

---

Key takeaways

• Old ReplicaSets kept at zero replicas are the rollback. Undo scales one back up.
• A rollback is a rolling update: it respects surge, unavailability and readiness, so it is not instant.
• It creates a new revision; history is preserved rather than rewritten.
• revisionHistoryLimit bounds how far back you can go. Setting it to 0 removes rollback entirely.
• Set kubernetes.io/change-cause or your history is a list of numbers when you most need detail.
• Rollback does not undo migrations, deleted data, or anything outside this Deployment.
• A readiness probe with maxUnavailable: 0 means a broken release stalls instead of shipping.

---

Check your understanding

---

Frequently Asked Questions

How do I roll back a Kubernetes deployment?

kubectl rollout undo deployment/<name> reverts to the previous revision, and --to-revision=N targets a specific one listed by kubectl rollout history. Check what a revision contains first with kubectl rollout history deployment/<name> --revision=N. The rollback runs as a normal rolling update, so it honours maxSurge, maxUnavailable and readiness probes. Watch it complete with kubectl rollout status.

Why is the CHANGE-CAUSE column empty?

Because nothing set it. It reads from the kubernetes.io/change-cause annotation, which Kubernetes does not populate automatically. Add it to the manifest's metadata, or run kubectl annotate deployment/<name> kubernetes.io/change-cause="..." --overwrite as part of your deploy. Having CI fill it from the commit message or release tag costs one line and turns an unusable list of revision numbers into something you can act on during an incident.

How many revisions does Kubernetes keep?

Ten by default, controlled by revisionHistoryLimit. Older ReplicaSets beyond that are deleted and their revisions can no longer be rolled back to. The default is sensible for most teams; setting it to 0 disables rollback entirely, which sometimes appears in manifests intended to reduce clutter and quietly removes the fastest recovery path available. Retained ReplicaSets sit at zero replicas and consume no compute.

Does rolling back undo database migrations?

No. A rollback changes which pod template is running and touches nothing else, no databases, no ConfigMaps, no Secrets, no external state. If the bad release applied a destructive migration, the older code will meet a schema it does not understand and may fail worse than before. The standard mitigation is expand-and-contract: ship schema changes that both the old and new versions tolerate, and only remove the old shape after the new version has proven itself.

Should I roll back or roll forward?

Roll back when the previous version is known good and the failure is clear. It restores a known state quickly and buys time to investigate properly. Roll forward when the new release has already changed persistent state in a way the old version cannot handle, when the previous version has its own known problem, or when the fix is genuinely small and well understood. When genuinely unsure, roll back; debugging under production pressure is worse than the delay.

Can I prevent bad deployments from reaching users at all?

Largely, yes. With a correct readiness probe and maxUnavailable: 0, a version that never becomes Ready never displaces the old Pods. The rollout stalls and fails after progressDeadlineSeconds while the application stays up. That converts a bad release into a failed deploy rather than an incident. Lowering progressDeadlineSeconds from its 600-second default makes the failure visible sooner, and kubectl diff -f before applying catches unintended changes.