What a Cache Is

Start in your kitchen

Start in your kitchen. The milk lives at the supermarket, twenty minutes away, but there is milk in your fridge because last time you went you brought some home.

Your fridge is a cache. Look at what is already true about it. The supermarket still has all the milk, so if your fridge runs empty you go back and nothing is lost. The fridge is small, so you keep only what you actually use. And milk goes off, so what is in your fridge can be older than what is on the shelf.

A copy, a source of truth behind it, limited space, and the copy going stale. That is the whole topic, and you understood it before anyone said the word cache.

The software version

Now the software version. A cache is a copy of data kept somewhere faster to reach than the original. Your product still lives in your database, which stays the truth. The cache holds a duplicate you can reach more quickly. Redis, the tool most people use for this, is a program that holds data in memory and hands it back by name.

Memory is faster for a reason worth knowing, because that answer is not a detail. Your database keeps data on a disk, and to read a row it works out where on that disk the row physically sits, then goes there and fetches it. Memory has no "where" to find. The machine addresses any byte directly. That is why the two differ by roughly a hundred times, and it comes from the hardware rather than from a decision anyone made.

Two words carry the rest of this course. A hit is a lookup that finds what it wanted. A miss is one that does not, and has to fall back to the database.

Two requests show both. The first asks the cache for product 8871. Nothing is there, so it reads the database, waits 27 milliseconds, and puts a copy in the cache before replying. The second request asks for the same product a moment later. This time the copy is there, so it replies in under a millisecond and never touches the database at all.

Everything else is detail on top of those two requests.

// Ask for a key. You get null on a miss, which is normal, not
// an error: it just means nobody has put it there yet.
String copy = redis.get("product:8871");

// Store a copy, with a time limit on how long it may be reused.
// That limit is called a TTL, for time to live.
redis.setex("product:8871", 300, json);   // 300 seconds

Worked example

Arjun timed both paths on the same product during peak traffic. His database returned product 8871 in 27 milliseconds. Redis, holding the same product as ready-made text, returned it in 0.4.

Then he worked out what that was worth. At 3,000 requests per second, if 95 out of every 100 are answered by the cache, his database sees 150 per second instead of 3,000. That is not five percent off the load. It is twenty times less of it.