Caching Patterns and Eviction Policies

A cache keeps frequently read data in fast memory close to the app; cache-aside, read-through, write-through and write-back decide how cache and database stay in sync, and eviction policies such as LRU, LFU and FIFO decide what to drop when memory is full.

A cache stores copies of data that is expensive to fetch or compute, in a faster layer. Done well it cuts latency and database load by an order of magnitude; done badly it serves wrong data.

flowchart LR
  APP["App"] -->|"1. GET key"| C["Cache (Redis)"]
  C -->|"hit"| APP
  C -.->|"2. miss"| APP
  APP -->|"3. query"| DB["Database"]
  APP -->|"4. SET key, TTL"| C
  %% step APP: The app asks the cache first.
  %% step C: On a hit, the answer comes from memory in about a millisecond.
  %% step DB: On a miss, the app reads the database.
  %% step C: The app stores the result in the cache with a time-to-live for next time.

Where caches live

Browser and CDN (module end), reverse proxy (Nginx), application cache (Redis, Memcached), database buffer cache, and in-process memory. A distributed cache is shared by all app instances, so they see the same cached data and the cache scales on its own nodes.

Patterns

Pattern	Read	Write	Good for
Cache-aside (lazy loading)	App checks cache, loads DB on miss	App writes DB, then deletes or updates the key	Most read-heavy apps
Read-through	Cache loads from DB itself	—	Libraries/proxies that support it
Write-through	—	Write cache and DB together	Data read right after writing
Write-back (write-behind)	—	Write cache now, DB later	Very high write rates; risk of loss

Eviction policies (when memory is full)

Policy	Drops	Note
LRU – least recently used	The item not used for the longest time	Good default
LFU – least frequently used	The item used least often	Keeps long-term popular items
FIFO – first in, first out	The oldest inserted item	Simple, ignores popularity
LIFO – last in, first out	The newest item	Rarely what you want
TTL-based	Items past their expiry	Combine with any of the above

Redis: maxmemory-policy allkeys-lru (or allkeys-lfu, volatile-ttl). With noeviction, writes fail when memory is full.

Why not write everything to the cache?

Caches are optimised for speed, databases for durability and correctness: transactions, constraints, indexes, queries, backups. Memory is expensive and volatile; a cache node restart can lose data. Use the cache as a copy, the database as the source of truth.

The hard parts

Invalidation: stale data after updates — delete the key on write and use TTLs.
Stampede: a hot key expires and 1,000 requests hit the database — use locks, jitter on TTLs, or refresh ahead.
Hit ratio: measure it; below ~80% for a read cache, check keys and TTLs.

Traps: caching per-user data under a shared key; no TTL at all; caching errors; noeviction on a cache that fills up.