Data Structures Beyond Strings

Structures, not strings

Memcached stores strings and that is the whole product. Redis stores structures.

See why that matters more than it sounds. Picking the right structure moves work out of your application into one atomic server-side command, and the alternative is a read, modify and write that races other writers.

Keep strings as your workhorse for cache values and counters, where atomic arithmetic is the backbone of most rate limiters.

Use a hash when you want a map of fields under one key. A session becomes fields you update one at a time, instead of rewriting a whole document and racing whoever else is writing it.

Take a list as a queue with two ends. Push on one, pop on the other, and the blocking pop lets your workers wait for jobs without polling. Half the lightweight job queues in production are exactly this.

Use a set for unique members with instant membership checks and server-side intersection. Intersecting two follower sets computes mutual friends without shipping either one over the network.

The sorted set

Learn the sorted set properly, because it is the star. Every member carries a score and stays ordered, with fast inserts and rank queries.

Watch what falls out of that. Bump a score by fifty, ask for the top ten, or find any player's position among millions, all in microseconds. Building it on a relational table means an index on score and constant re-sorting under write load, and here it is the native operation.

Use the same structure for a sliding-window rate limiter, with timestamps as the scores and a trim to the window.

Round out the set with two more. One counts unique things, daily visitors say, in 12 kilobytes with under a percent of error however many there are. The other gives you an append-only log with consumer groups, closer to a small message broker than to broadcasting.

Take the interview signal for what it is: matching your structure to your access pattern, rather than reaching for a string and encoding a document into it.

// Counter: atomic, no read-modify-write race, no lock.
redis.incr("views:page:42");

// Hash: update one field without rewriting the whole session.
redis.hset("session:8812", "cartCount", "3");

// Sorted set: a leaderboard that stays ordered as you write to it.
redis.zincrby("leaderboard", 50, "player:9");
redis.zrevrange("leaderboard", 0, 9);        // top ten, in microseconds

// Sorted set again, as a sliding-window rate limiter.
long now = System.currentTimeMillis();
redis.zremrangeByScore("rate:user:7", 0, now - 60_000);   // drop the old
long used = redis.zcard("rate:user:7");
if (used < 100) redis.zadd("rate:user:7", now, now + ":" + rand());

Worked example

Yuki is building the leaderboard for a mobile puzzle game with 8 million weekly players. Version one is Postgres: a scores table, an index on score, and a rank query using COUNT(*) of players with higher scores, which takes 400 ms at peak and gets slower as the season progresses. Players see their rank on every game-over screen, so this query runs 2,000 times a second. Version two is one Redis sorted set per season. Score updates become ZINCRBY, the top-100 screen is ZREVRANGE with scores, and a player's own rank is ZREVRANK, each returning in under a millisecond. The whole leaderboard service shrinks to about 200 lines, and the season-end snapshot job just walks the sorted set once and archives it to Postgres, where historical queries belong anyway.