DevOps PeachDrift happens when someone changes infrastructure outside Terraform: a hot-fix in the console, a script, another tool. Left alone, the next unrelated apply silently reverts the hot-fix, or fails.
flowchart LR
SCH["Nightly CI: plan -detailed-exitcode"] --> EX{"exit code"}
EX -->|"0"| OK["No drift"]
EX -->|"2"| ALERT["Drift: open an issue"]
ALERT --> DEC{"Keep the change?"}
DEC -->|"yes"| CODE["Update code, or apply -refresh-only"]
DEC -->|"no"| REV["terraform apply: revert"]
%% step SCH: A scheduled pipeline runs plan against every environment, for example every night.
%% step EX: With -detailed-exitcode, plan exits 0 for no changes, 1 for errors and 2 for changes.
%% step ALERT: Exit code 2 means code and reality differ, so the job alerts the owners.
%% step DEC: A person decides whether the outside change was right.
%% step CODE: If it was right, put it in the code (or accept it into state) so Terraform stops reverting it.
%% step REV: If not, a normal apply puts the infrastructure back to the code.
Seeing drift
| Command | Shows |
|---|---|
terraform plan -refresh-only |
Only differences between state and reality |
terraform plan -detailed-exitcode |
Exit 0 = no changes, 1 = error, 2 = changes present (for scripts) |
terraform apply -refresh-only |
Accept reality into state without changing infrastructure |
terraform plan -detailed-exitcode -input=false -lock=false
case $? in
0) echo "clean" ;;
2) echo "drift or pending changes" ; exit 2 ;;
*) echo "plan failed" ; exit 1 ;;
esac
Deciding what to do
| Situation | Action |
|---|---|
| Hot-fix that should stay | Change the code to match, then plan shows nothing |
| Change by a tool that owns that field | lifecycle { ignore_changes = [...] } |
| Unwanted manual change | terraform apply reverts it |
| Object is broken but config is fine | terraform apply -replace='docker_container.app["web"]' recreates it (replaces the old taint) |
check blocks: continuous assertions
check "shop_api_health" {
data "http" "health" {
url = "http://localhost:8081/healthz"
}
assert {
condition = data.http.health.status_code == 200
error_message = "shop-api health check returned ${data.http.health.status_code}."
}
}
A check runs on every plan and apply. A failing assertion is a warning, not an error, so it never blocks a deployment; it is ideal for "is the service still answering?" checks. The data source inside it is scoped to the check. HCP Terraform can run these checks on a schedule (health assessments).
Prevent drift in the first place
- Give humans read-only access to environments that Terraform manages.
- Make the pipeline the only way to apply.
- Use policies or alerts on console changes to managed resources.
Traps: -replace only plans the replacement of the given address; dependent resources may also change, so read the plan; and ignore_changes = all hides every future change to the resource, including the ones you want.