DevOps Peach

System Design for DevOps · Communication and Asynchronous Processing · Lesson 3 of 4

API Design: Idempotency, Pagination and Rate Limiting

Good APIs are predictable under failure and load: idempotency keys make retries safe, cursor pagination keeps large lists fast, versioning avoids breaking clients, and rate limiting protects the service with clear 429 responses.

Practice interactively →

An API is a contract that other teams and customers build on. In distributed systems it must also behave well when the network fails and when clients misbehave.

flowchart LR
  C["Client"] -->|"POST /v1/orders + Idempotency-Key"| GW["API gateway"]
  GW --> RL{"Within rate limit?"}
  RL -->|"no"| R429["429 Too Many Requests + Retry-After"]
  RL -->|"yes"| SVC["orders service"]
  SVC --> KEY{"Key seen before?"}
  KEY -->|"yes"| SAME["Return the stored response"]
  KEY -->|"no"| NEW["Create order, store response"]
  %% step C: Clients may retry after a timeout without knowing whether the first try worked.
  %% step GW: The gateway authenticates, versions and limits traffic.
  %% step RL: A token bucket per client decides whether to accept the request.
  %% step SVC: The service checks the idempotency key before doing the work.
  %% step SAME: A retry with the same key gets the original result — no duplicate order.

REST, gRPC, GraphQL

Style Good for Note
REST/JSON Public APIs, browsers Resources + HTTP verbs; cacheable GETs
gRPC Service-to-service Fast binary protocol, streaming, generated clients; needs HTTP/2
GraphQL Many clients needing different shapes One endpoint; harder caching and rate limiting

Idempotency

GET, PUT and DELETE are idempotent by definition; POST is not. A client that times out and retries POST /orders may create two orders. Fix: the client sends an Idempotency-Key header (a UUID); the server stores the key with the result and returns the same result for repeats (for example for 24 hours).

Pagination

Type How Problem it avoids
Offset ?offset=10000&limit=50 Simple, but slow deep pages; items shift when data changes
Cursor ?limit=50&cursor=eyJpZCI6MTAwNTB9 Fast at any depth, stable under inserts

Versioning and compatibility

Add fields freely; never remove or rename fields in place. Use /v1/ or a header, and run old and new versions side by side during migration.

Rate limiting

Algorithm Behaviour
Fixed window N requests per minute; bursts at window edges
Sliding window Smoother count over the last minute
Token bucket Allows short bursts up to the bucket size, steady refill rate

Respond with 429 and Retry-After; publish limits in headers (RateLimit-Limit, RateLimit-Remaining). Limit per API key or user, and keep counters in a shared store (Redis) so all instances agree.

Timeouts and errors

Every call needs a timeout. Return clear status codes: 400 for bad input, 404 not found, 409 conflict, 429 limited, 503 temporarily unavailable — clients retry only 429/503 and network errors, with backoff (module 5).

Traps: non-idempotent POST with automatic retries; offset pagination on million-row tables; breaking field renames; per-instance rate-limit counters that multiply the limit by the number of pods.

Practice interactively →