DevOps Peach

Advanced Terraform · State and Teamwork · Lesson 2 of 4

Remote Backends and State Locking

A backend stores state remotely; the S3 backend locks with a .tflock object (use_lockfile), takes environment-specific settings through partial configuration and migrates local state with init.

Practice interactively →

A backend decides where state lives and how it is locked. Locally it is a file; for a team it must be shared, encrypted, versioned and locked so two applies cannot run at the same time.

flowchart LR
  D1["Developer A: apply"] --> LK["Lock: dev/app.tfstate.tflock"]
  D2["Developer B: apply"] -.->|"waits / fails"| LK
  LK --> S3["S3 bucket shop-tfstate (versioned, encrypted)"]
  CI["CI pipeline"] --> LK
  LS["LocalStack :4566"] --- S3
  %% step D1: The first run that starts writes a lock object next to the state.
  %% step D2: A second run sees the lock and waits for -lock-timeout or stops with an error.
  %% step LK: With use_lockfile = true the S3 backend locks with a .tflock object; no DynamoDB table is needed.
  %% step S3: Bucket versioning keeps old states so you can roll back a bad write.
  %% step LS: In this course LocalStack emulates S3 on your laptop, so nothing costs money.

An S3 backend on LocalStack

terraform {
  backend "s3" {
    bucket         = "shop-tfstate"
    key            = "dev/app.tfstate"
    region         = "us-east-1"
    encrypt        = true
    use_lockfile   = true                      # S3-native locking (Terraform 1.10+)
    use_path_style = true
    endpoints      = { s3 = "http://localhost.localstack.cloud:4566" }
    skip_credentials_validation = true
    skip_requesting_account_id  = true
  }
}

Credentials never go in this block: export AWS_ACCESS_KEY_ID=test and AWS_SECRET_ACCESS_KEY=test for LocalStack, or use a profile or OIDC role in real AWS. Older setups lock with a DynamoDB table (dynamodb_table); that option is deprecated in favour of use_lockfile.

Other backends work the same way: azurerm (blob lease), gcs (lock object), HCP Terraform (cloud block, with runs and policies on the server).

Partial configuration

Backend blocks cannot use variables. Keep the shared part in code and pass the rest at init:

terraform init -backend-config=backend/dev.s3.tfbackend
# backend/dev.s3.tfbackend
#   key = "dev/app.tfstate"

Moving and changing backends

Command When
terraform init -migrate-state You added or changed the backend block and want to copy the existing state over
terraform init -reconfigure Use the new backend settings and ignore the old ones (no copy)
terraform plan -lock-timeout=5m Wait up to five minutes for another run's lock
terraform force-unlock <LOCK_ID> Remove a lock left by a crashed run, only after you checked nobody is applying

Bootstrapping

The bucket that holds state must exist before the backend can use it. Create it once with a small separate configuration that keeps its own state locally (or with a script), enable versioning and encryption, and never manage it from the configuration that stores state in it.

Traps: force-unlock while another apply is really running can corrupt state; and -reconfigure after changing the key leaves you with an empty state, so the next plan wants to create everything again.

Practice interactively →