System Design for DevOps · Module 02

Scalability

Grow a system with demand: scale up or out, spread traffic with load balancers and health checks, keep services stateless so any instance can serve any request, scale automatically with metrics, and place data on nodes with consistent hashing.

Practice interactively →

What you will learn

• Choose between vertical and horizontal scaling and know the limits of each
• Configure load balancing algorithms, health checks and session handling
• Make services stateless and scale them automatically on the right metric
• Explain consistent hashing and why it beats modulo hashing when nodes change

Lessons

1. Vertical vs Horizontal Scaling

   Vertical scaling adds CPU and memory to one machine; horizontal scaling adds more machines behind a load balancer. Vertical is simple but limited and still a single point of failure; horizontal needs stateless design and data consistency care.

2. Load Balancers, Algorithms and Health Checks

   Load balancers spread requests across healthy instances: L4 balancers route TCP connections, L7 balancers understand HTTP; algorithms such as round robin and least connections pick a target, and health checks remove failed ones.

3. Stateless Services and Session Handling

   A stateless service keeps no client data between requests, so any instance can serve any request; sessions move to a shared store such as Redis or into signed tokens, while sticky sessions are a fragile workaround.

4. Elasticity and Autoscaling

   Elasticity is scaling out and back in automatically with demand; autoscalers such as the Kubernetes HPA, cluster autoscaler and cloud Auto Scaling groups react to metrics like CPU, requests per target or queue depth.

5. Consistent Hashing

   Consistent hashing places nodes and keys on a ring so that adding or removing a node moves only about 1/N of the keys, unlike modulo hashing which remaps almost everything; virtual nodes even out the load.