DevOps PeachThe method for every case: 1) requirements, 2) estimates, 3) API, 4) data model, 5) architecture, 6) bottlenecks and scaling.
flowchart LR U["User clicks sho.rt/aZ3k9Q"] --> CDN["CDN / edge"] CDN --> LB["Load balancer"] LB --> API["Redirect service"] API --> C["Redis cache"] API --> KV["Key-value store: code → URL"] API -.->|"click event"| Q["Queue → analytics"] W["POST /shorten"] --> API %% step U: Every click is a GET on a short code, answered with an HTTP redirect. %% step CDN: Popular redirects can even be cached at the edge. %% step API: Stateless redirect service, scaled horizontally behind the load balancer. %% step C: Hot codes are served from Redis — most clicks never reach the database. %% step KV: A key-value store holds code → long URL; lookups by key only. %% step Q: Click analytics are sent asynchronously so they never slow down the redirect.
1. Requirements
- Functional: create a short link (optional custom alias, expiry); redirect; basic click stats.
- Non-functional: redirect p99 < 50 ms, 99.99% available for redirects, links never collide, codes not easily guessable.
2. Estimates
100 M new links/month ≈ 40 writes/s. Read:write 100:1 → ~4,000 redirects/s average, ~20,000 peak. 5 years × 1.2 B links × ~500 bytes ≈ 3 TB. Cache the hottest 20% of daily traffic: a few GB of Redis.
3. API
POST /v1/links {"url": "...", "alias": "optional", "expires_at": "..."} → 201 {"code": "aZ3k9Q"}
GET /{code} → 301/302 Location: <long url>
GET /v1/links/{code}/stats
301 (permanent) lets browsers cache the redirect — fewer requests but less accurate stats. 302 keeps every click visible.
4. Data model
links(code PK, long_url, user_id, created_at, expires_at) in a key-value store (DynamoDB, Cassandra) — every lookup is by key, and it scales out.
5. Generating codes
7 characters of base62 = 62⁷ ≈ 3.5 trillion codes.
| Approach | Pros | Cons |
|---|---|---|
| Hash of URL (first 7 chars) | Same URL → same code | Collisions to handle |
| Counter + base62 (ID ranges per server) | No collisions, fast | Sequential codes are guessable → shuffle/encrypt the number |
| Random + check exists | Simple, unguessable | Extra read per write; rare retries |
A common answer: each app server reserves a block of 1,000 IDs from a central counter (Redis INCRBY or a database sequence), then encodes them in base62.
6. Bottlenecks
- Reads: cache-aside in Redis (LRU), CDN for very hot links, replicas of the KV store.
- Hot keys: a viral link → local in-memory cache in each redirect pod for a few seconds.
- Analytics: never write a row per click synchronously — publish an event to a queue/stream and aggregate later.
- Abuse: rate limit creation per user/IP, scan URLs for malware.
Traps: a relational database with joins for a pure key lookup; synchronous click counting; sequential guessable codes; forgetting expiry and cleanup.