StatefulSets and headless Services

A StatefulSet gives each pod a stable identity — ordered name, own DNS, own PVC — for stateful applications.

Unlike a ReplicaSet (random, interchangeable pod names), a StatefulSet keeps an identity for each pod: hostname, DNS and storage stay stable across recreation. Use it for databases, queues and clusters such as Cassandra or Kafka.

apiVersion: v1
kind: Service
metadata: { name: training }
spec:
  clusterIP: None          # headless
  selector: { app: training }
  ports: [{ port: 80 }]
---
apiVersion: apps/v1
kind: StatefulSet
metadata: { name: training }
spec:
  serviceName: training
  replicas: 2
  selector: { matchLabels: { app: training } }
  template:
    metadata: { labels: { app: training } }
    spec:
      containers:
      - { name: web, image: nginx:1.27, volumeMounts: [{ name: data, mountPath: /usr/share/nginx/html }] }
  volumeClaimTemplates:
  - metadata: { name: data }
    spec: { accessModes: [ReadWriteOnce], resources: { requests: { storage: 1Gi } } }

flowchart LR
  P0["training-0"] --- V0[("data-training-0")]
  P1["training-1"] --- V1[("data-training-1")]
  P2["training-2"] --- V2[("data-training-2")]

Pod names are <sts>-<ordinal>: training-0, training-1. They are created in order (the next only after the previous is Ready) and deleted in reverse order.
A headless Service (clusterIP: None) has no virtual IP; DNS returns each pod IP, and every pod gets a name like training-0.training.default.svc.cluster.local.
volumeClaimTemplates creates one PVC per pod: data-training-0… PVCs are not deleted when the StatefulSet is deleted or scaled down, so data is not lost.
updateStrategy.rollingUpdate.partition: 2 only updates pods with ordinal ≥ 2, highest first.

Command	Meaning
`kubectl scale sts training --replicas=3`	Scale a StatefulSet
`kubectl get pvc`	Each pod's PVC
`kubectl get pods -l app=training`	`training-0`, `training-1`…

Pod management and updates

Setting	Effect
`podManagementPolicy: OrderedReady` (default)	Create, scale, and delete one pod at a time in order
`podManagementPolicy: Parallel`	Start or stop pods together, when the app does not need ordering
`updateStrategy: RollingUpdate`	Update from the highest ordinal down, waiting for each to be ready
`updateStrategy: OnDelete`	Only update a pod when you delete it yourself
`persistentVolumeClaimRetentionPolicy`	Choose whether PVCs are deleted when the StatefulSet is deleted or scaled down

The partition field enables staged rollouts: set it to the replica count minus one, update, check training-2, then lower the partition step by step.

Stable identity in practice

Clustered software uses the stable names to find peers. A three-node database can be configured with db-0.db, db-1.db, and db-2.db as seed addresses because those names survive restarts. Each pod's volume follows it to whichever node it is rescheduled on, as long as the storage can attach there (zone limits apply to cloud disks).

What a StatefulSet does not do

It does not make an application highly available by itself. Replication between members, leader election, backups, and failover are the application's job, or an operator's. Running a production database means planning backups and restore tests, not just writing a StatefulSet. Many teams prefer a managed database service, or a mature operator, over a hand-written StatefulSet.

Debugging tips

A pod stuck Pending usually has an unbound PVC: check kubectl get pvc and the StorageClass.
If training-0 never becomes ready, training-1 will not be created under OrderedReady, so fix the first pod first.
kubectl delete pod training-1 is safe: the same name and volume come back.

Deleting a StatefulSet does not guarantee pods terminate in order; scale it to 0 first when order matters.