Load Balancers, Algorithms and Health Checks

Load balancers spread requests across healthy instances: L4 balancers route TCP connections, L7 balancers understand HTTP; algorithms such as round robin and least connections pick a target, and health checks remove failed ones.

A load balancer (LB) is the front door of a horizontally scaled tier. It hides individual instances and removes the broken ones.

flowchart LR
  C["Clients"] --> LB["Load balancer"]
  LB -->|"healthy"| A["target 1"]
  LB -->|"healthy"| B["target 2"]
  LB -.->|"failed health check"| X["target 3"]
  HC["Health checks: GET /healthz"] -.-> A
  HC -.-> X
  %% step C: Clients use one DNS name; they never see individual servers.
  %% step LB: The balancer picks a target for each connection or request.
  %% step A: Healthy targets receive traffic according to the algorithm.
  %% step X: A target that fails its health check is taken out until it passes again.
  %% step HC: Health checks run every few seconds against a cheap endpoint.

Layer 4 vs layer 7

	L4 (transport)	L7 (application)
Sees	IP, port, TCP/UDP	HTTP method, path, headers, cookies
Can route by	Port	Path (`/api`), host, header
TLS	Pass-through or terminate	Terminates TLS, can add WAF
Examples	AWS NLB, HAProxy TCP mode	AWS ALB, Nginx, Envoy, Ingress controllers
Speed	Very fast, millions of connections	More features, a bit more latency

Algorithms

Algorithm	Picks	Good when
Round robin	Next target in order	Similar requests, similar servers
Weighted round robin	More often for bigger targets	Mixed instance sizes, canary 10%
Least connections / least outstanding requests	Target with fewest active requests	Requests of very different duration
IP hash / consistent hash	Same target for the same key	Cache affinity

Health checks

Point them at a cheap endpoint (/healthz) that checks the process, not every dependency — otherwise one slow database marks all targets unhealthy.
Tune interval, timeout and thresholds: e.g. every 10 s, unhealthy after 2 failures, healthy after 3 successes.
Connection draining (deregistration delay) lets in-flight requests finish before a target is removed during deploys.

The LB itself

A single self-managed LB is a SPOF. Managed balancers (ALB, NLB, Azure Load Balancer, Google Cloud LB) run across zones. Self-managed ones need a pair with a floating IP (keepalived) or DNS failover.

Traps: health checks that call the database; no draining, so every deploy drops requests; sticky sessions hiding a stateful app (next lesson); one LB in one zone.