Four places a copy can sit

The copy can sit in four places, because a cache is not one thing in one place. There are four answers, and knowing all four is what lets you say "there are a few options here" before picking one.

Closest to the user is their own device. Their browser or phone can keep the copy itself, which is the fastest possible cache because a hit costs no network traffic at all. It is also the one you control least. You cannot reach into a million phones to correct a copy, so your only lever is the time limit you set when you sent it.

Next is a CDN, short for content delivery network. It is a set of servers the provider runs in cities all over the world, each holding copies of your files. The win here is distance. A user in Pune fetching an image from a server in Virginia waits about 250 milliseconds for it to cross the planet. The same image from a server in Mumbai takes 20.

Third is inside your own application, in the server's own memory. No network is involved, so it is faster than Redis. The catch is that each of your servers has its own copy, and they do not talk to each other. Change something on one and the other nine carry on serving the old value. Keep this for small things that rarely change, like configuration or a currency table.

Last is a shared cache, a separate service such as Redis that all your servers talk to. It costs a network round trip, so it is slower than keeping the copy in your own memory. You will still reach for it most, because it is the only option where every server sees the same data and deleting a key corrects all of them at once.

Reach for the shared cache first and treat the others as refinements. Add a CDN when you are serving images or video. Add an in-process copy only when one key is so busy that even Redis is straining under it.

the shape of it

step 1 of 4

Each layer answers what it can. A request that reaches the database has missed four copies on the way.

your own memory, against a shared cache

Java

// In your server's memory. Fastest, and private to this one
// server, so the other nine will not see your change.
private final Map<String, Config> local = new ConcurrentHashMap<>();

Config getFlag(String name) {
  return local.computeIfAbsent(name, db::loadFlag);
}

// Shared. One round trip over the network, and every server
// sees the same copy. One delete fixes all of them.
String getProduct(long id) {
  return redis.get("product:" + id);
}

Worked example

Arjun ended up using three of the four. Product images went to a CDN, and load times for his users in India fell from 280 milliseconds to 30.

Product details went into Redis, shared by all 40 of his servers, so changing a price meant deleting one key and every server picked up the new one on its next request.

The currency table, forty rows that update once a day, went into each server's own memory. It was too small to be worth a network trip, and a day out of date was fine because the rates only changed daily anyway.

Where Caches Live

Four places a copy can sit

Worked example