Which eviction policy should I pick if I am not sure?

LRU. It matches the access patterns of most web workloads and it is the default in Redis and most caching libraries. Move to LFU only when you can show a stable set of very popular keys being evicted by bursts of one-time reads, and consider excluding scan-type traffic from the cache before switching policies.

Is adding TTL jitter really necessary if my keys are populated at different times?

Yes, because synchronized population sneaks in through deploys, cache flushes, warming scripts, and bulk jobs, all of which set thousands of keys within the same second. Jitter costs one line and a few percent of cache freshness, and it removes an entire class of outage that otherwise fires days after its cause.

How is a cache stampede different from a hot key problem?

A hot key is heavy read traffic on one key, which the cache handles fine while the entry is present; the pressure is on the cache node. A stampede is what happens at the moment that entry disappears, when the read traffic converts into duplicate database queries. Hot keys are a capacity concern; stampedes are a transition concern, and the defenses target the transition.

Cachingchapter 5 of 8 · 7 lessons

Cache Eviction & Stampede

What gets removed when cache is full, and how to handle mass expiry.

← All chapters View in cheatsheet

Your cache holds 8 gigabytes and your catalogue is 40. It has been full since Tuesday, so every new entry pushes an old one out.

Which one leaves is a rule you chose, possibly without reading it. Choose badly and the overnight job that walks all 200,000 products evicts the 300 that people actually buy, and your morning traffic arrives to an empty cache.

Then there is the other failure, the one that happens in a single second. One popular key expires, the 5,000 requests a second that were being served from memory all miss at once, and every one of them goes to the database together.

That database was comfortable a moment ago and is now taking 5,000 copies of the same query. It has a name worth knowing before you meet it at 3am.

Lessons

7 in this chapter

Eviction PoliciesLRU bets on recency, LFU bets on frequency, and TTL is not an eviction policy at all.2 min
LFU: Counting Instead of TimingEvict what is asked for least, not what was asked for longest ago, and survive the batch job that ruins LRU.2 min
FIFO, Random, and Why Nobody Picks ThemTwo policies that are simpler than LRU, worse than LRU, and still worth being able to dismiss.2 min
The Stampede ProblemWhen a hot key expires, every waiting request charges the database at once.2 min
Locks, Jitter, and Early RefreshLet one request do the work, stagger the expiries, or refresh before the deadline.3 min
Hot KeysOne key taking more traffic than a whole server can serve, and no amount of sharding helps.2 min
Negative CachingCache the fact that something does not exist, or misses for missing data will never end.3 min