DevOps Peach

Advanced Terraform · State and Teamwork · Lesson 4 of 4

Splitting State and Sharing Outputs

Split one big state into layers with a small blast radius, then pass values between them through outputs, terraform_remote_state or, better, data sources.

Practice interactively →

One state for everything is easy at first and painful later: every plan refreshes every resource, every apply can break everything, and everyone needs access to all of it. Split by lifecycle and ownership.

flowchart TB
  NET["network state (platform team)"] -->|"outputs: network_name"| APP["app state (shop team)"]
  DB["data state (DBA)"] -->|"outputs: db_host"| APP
  APP --> MON["monitoring state"]
  %% step NET: The network changes rarely and is owned by the platform team, so it gets its own state.
  %% step DB: Databases are long-lived and risky to touch, so they live apart from the app.
  %% step APP: The app layer changes many times a day and only needs a few values from the layers below.
  %% step MON: Each state has its own backend key, its own permissions and a small blast radius.

How to cut

Split by Example
Change rate Network (monthly) vs app (daily)
Owner Platform team vs product team
Blast radius A mistake in the app state cannot delete the database
Environment dev/app.tfstate, prod/app.tfstate (module 3)

Aim for states that plan in under a minute and fit one team.

Sharing values: three options

1. Data sources (preferred). Look the object up by name or tag from the provider; the layers stay decoupled.

data "docker_network" "shop" {
  name = "shop-dev"
}
# use data.docker_network.shop.id

2. terraform_remote_state. Read another state's root outputs:

data "terraform_remote_state" "network" {
  backend = "s3"
  config = {
    bucket = "shop-tfstate"
    key    = "dev/network.tfstate"
    region = "us-east-1"
  }
}
# use data.terraform_remote_state.network.outputs.network_name

Easy, but the reader needs read access to the whole other state, secrets included, and the layers are tightly coupled.

3. A parameter store. The producer writes values to SSM Parameter Store, Consul or Vault; consumers read them with a data source. Good across teams and tools.

Order of applies

Separate states have no automatic dependency graph between them. Apply network, then data, then app; a CI pipeline or a tool such as Terragrunt (module 5) runs them in order.

Traps: terraform_remote_state only sees root outputs, so a child module's output must be re-exported by the root; and splitting an existing state means moving resources between states, which you do with import blocks in the new state plus removed { lifecycle { destroy = false } } in the old one.

Practice interactively →