Terraform State and Backends
Why Terraform keeps state, what lives in it, how remote backends and locking make teamwork safe, and the state commands that get you out of trouble.
State is the concept that separates people who find Terraform predictable from people who find it mysterious. Almost every confusing Terraform situation. A resource it wants to recreate for no obvious reason, a change made in the console that gets undone, two people applying at once and breaking everything. Traces back to it.
Why state exists
Terraform needs to know that aws_instance.web in your configuration is instance i-0abc123 in AWS. Nothing in the configuration says so, and nothing in AWS records it. That mapping is what state holds.
Without it, Terraform could not tell the difference between "create this instance" and "update the instance I made last week", so every apply would create duplicates.
State does three jobs:
| Job | Why it matters |
|---|---|
| Maps config to real resources | Turns aws_instance.web into i-0abc123 |
| Caches attributes | Plans can be produced without querying every API every time |
| Records dependencies | Destroys happen in the correct order even after resources are removed from config |
What is actually in the file
terraform.tfstate is JSON: a serial number, the Terraform version, and an array of resource instances with every attribute the provider returned.
That last part is the important one. State contains resource attributes in plain text, which includes generated passwords, private keys, and anything passed through a variable marked sensitive. The sensitive flag redacts console output. It does nothing to state.
So:
- Never commit state to version control.
- Encrypt it at rest and restrict who can read it.
- Treat a state file as one of the most valuable artifacts in the repository.
Drift
Terraform refreshes state before planning, querying providers for current attributes, and compares that against what it recorded. When someone changes something in a console, the plan shows Terraform intending to change it back.
That is why an untouched configuration can produce a non-empty plan. It is not a bug; it is Terraform reporting that reality and code disagree.
terraform plan -refresh-only shows drift without proposing configuration changes alongside it, which is the clean way to review what has been altered out of band.
Local state does not survive contact with a team
A local terraform.tfstate works exactly until a second person or a CI pipeline appears. Then:
- It lives on one laptop and cannot be shared.
- There is no version history, so a bad write is unrecoverable.
- Two people applying simultaneously produce a corrupted file.
- Losing the machine loses the record of your infrastructure.
A remote backend solves all four.
Remote backends
A backend defines where state lives and how it is locked:
terraform {
backend "s3" {
bucket = "acme-tfstate"
key = "production/terraform.tfstate"
region = "eu-west-1"
encrypt = true
use_lockfile = true
}
}
The S3 backend with locking is the most common AWS pattern. Modern Terraform supports native S3 locking via use_lockfile; older versions require a DynamoDB table referenced with dynamodb_table.
Three things to get right:
Enable bucket versioning. This is the difference between recovering from a bad state write in minutes and rebuilding an environment.
Separate state per environment. Give each its own key, production/terraform.tfstate, staging/terraform.tfstate. A mistake in staging then cannot touch production state.
Bootstrap the bucket separately. The bucket must exist before terraform init can use it, so it cannot be created by the configuration that stores state in it. Create it in a small separate configuration with local state, or once via the CLI.
Backend configuration cannot use variables, so supply differing values at init time:
terraform init -backend-config="key=production/terraform.tfstate"
Locking
When Terraform starts an apply it acquires a lock, and releases it when finished. A second apply waits or fails rather than writing over the first. This is the single most important reason to use a remote backend in a team.
If a process is killed mid-apply, the lock can stick:
terraform force-unlock <LOCK_ID>
Confirm nobody is actually running an apply first. Force-unlocking a live apply is exactly the corruption locking exists to prevent.
The state commands
These edit state directly and bypass the safety of a plan. Take a backup before using them.
| Command | Use |
|---|---|
terraform state list | What does Terraform manage? Read-only, always safe |
terraform state show <addr> | What does Terraform think this looks like? Read-only |
terraform state mv | Rename or move without destroying |
terraform state rm | Stop managing something without destroying it |
terraform import | Adopt existing infrastructure |
The two read-only ones answer most confusing plans in seconds. Reach for them before anything else.
For renames and moves, prefer moved blocks in configuration over state mv. They are declarative, reviewable in a pull request, and applied by a normal plan:
moved {
from = aws_instance.web
to = module.compute.aws_instance.web
}
What to do if state is lost
Restore from backend versioning, which is why it should be enabled before you need it. Failing that, the recovery path is importing every resource back into a fresh state. Slow and error-prone for anything non-trivial.
If state contains a resource that no longer exists, terraform state rm brings it back in line. If real infrastructure exists that state does not know about, terraform import adopts it.
Explore further in this section
| Sub-topic | What you will learn |
|---|---|
| State Overview | What state contains, drift, and security concerns |
| Remote Backends | S3 and locking setup, migration, bootstrapping |
| State Commands | mv, rm, import and refactoring safely |
Quick reference
| Task | Command |
|---|---|
| List managed resources | terraform state list |
| Inspect one | terraform state show <address> |
| Review drift only | terraform plan -refresh-only |
| Rename without recreating | terraform state mv <old> <new> |
| Stop managing | terraform state rm <address> |
| Adopt existing infra | terraform import <address> <id> |
| Clear a stuck lock | terraform force-unlock <LOCK_ID> |
| Migrate backends | edit backend block, then terraform init |
Final Thoughts
State is not an implementation detail you can ignore. It is the thing Terraform reasons from, and understanding it converts most Terraform confusion into something obvious. Set up a remote backend with locking and versioning before you need it, keep environments in separate state files, and never let state into version control.
Frequently Asked Questions
What is Terraform state and why can't Terraform work without it?
State is Terraform's record of which real-world resources correspond to which configuration blocks. Nothing in your code or in the cloud provider records that aws_instance.web is i-0abc123. State does. Without it Terraform cannot distinguish creating a resource from updating one it made previously, so every apply would create duplicates. State also caches attributes so plans are fast, and records dependencies so destroys happen in the right order.
Is it safe to commit the Terraform state file to Git?
No. State stores resource attributes in plain text, including generated passwords, private keys and anything passed through a variable marked sensitive, that flag only affects console output. Committing it exposes secrets to everyone with repository access and to Git history permanently. It also does not solve the problems a backend solves, since Git provides no locking, so two people can still apply simultaneously. Use a remote backend with encryption and restricted access.
What is a Terraform backend?
The configuration that tells Terraform where to store state and how to lock it. The default is local, writing terraform.tfstate to the working directory. Remote backends such as S3, Azure Blob Storage, Google Cloud Storage or Terraform Cloud store it centrally so it can be shared, versioned and locked. Switching is a matter of adding a backend block and running terraform init, which offers to migrate the existing state.
Why does state locking matter?
Because two simultaneous applies writing to the same state file will corrupt it, leaving Terraform with an inaccurate record of your infrastructure, which is considerably harder to fix than the original problem. Locking makes the second run wait or fail instead. It becomes essential the moment more than one person or a CI pipeline can apply, and it is the main reason local state does not scale beyond one developer.
How do I separate state between dev, staging and production?
Give each environment its own state file, typically a different key in the same backend bucket. This means a mistake in one cannot corrupt another's state, environments can be applied independently, and credentials can be scoped per environment. Terraform workspaces offer a lighter alternative, but they share one backend and one set of credentials, which is why separate state files, and ideally separate directories, are preferred for production.
What should I do if terraform plan shows changes I did not make?
That is drift: someone changed the infrastructure outside Terraform, and the plan is proposing to bring it back in line with your configuration. Run terraform plan -refresh-only to review what changed without mixing it with configuration changes. Then decide whether to accept the manual change by updating your code, or let Terraform revert it. Leaving drift unresolved is how teams stop trusting Terraform, and a tool people distrust gets bypassed.