Fan-out on Write vs Fan-out on Read

Push

Push, and you fan out on write. When Dana posts, a worker reads her follower list and inserts the post identifier into every follower's feed, usually a sorted set in memory scored by time.

Get instant reads out of that. One range query, a few milliseconds, no matter how many accounts somebody follows.

Count the costs. Write amplification in proportion to follower count. Wasted work for followers who have not opened the app in months. And the celebrity problem, where one post from a fifty-million-follower account triggers fifty million writes and clogs the queue for everybody behind it.

Pull

Pull, and you fan out on read. Nothing happens when Dana posts beyond storing it. When a follower opens the app, your feed service fetches recent posts from each of the 400 accounts she follows, merges by time, and returns the top.

Notice what flipped. Your writes stay cheap and dormant users cost nothing, and every feed open now does hundreds of lookups, with your slow percentile hostage to the slowest shard on every single read.

Take each model where it is strong, the hybrid everybody actually ships. Accounts below a follower threshold fan out on write. Accounts above it do not fan out at all, and their posts simply sit in their own timeline.

Merge at read time. Take the precomputed feed, pull fresh posts from the handful of enormous accounts this person follows, merge, and return.

See why the pull side stays small. Anyone follows only a few million-strong accounts, usually under a dozen, and those timelines are so hot they are permanently cached anyway.

Say the threshold logic out loud in an interview, then note that it is tunable. The right cutoff falls out of your queue capacity and your read latency budget, not out of a universal constant.

void onPost(Post p) {
  long followers = graph.followerCount(p.authorId());

  if (followers < CELEBRITY_THRESHOLD) {
    // Push: one write per follower, reads become a single range query.
    for (long f : graph.followers(p.authorId())) {
      feeds.zadd("feed:" + f, p.createdAt(), p.id());   // ids, never bodies
      feeds.ztrim("feed:" + f, 800);
    }
  }
  // Above the threshold, do nothing. 50 million writes would still
  // be running an hour later with the queue backed up behind them.
}

List<Post> readFeed(long userId) {
  List<Long> ids = feeds.zrevrange("feed:" + userId, 0, 49);   // precomputed
  ids.addAll(recentPostsFrom(graph.celebritiesFollowedBy(userId)));  // merged in
  return hydrate(sortByTimeDesc(ids));      // bodies fetched in one batch
}

Worked example

Twitter's own engineers told this story publicly for years, and the numbers make it concrete. Around 2013 the platform saw about 5,000 tweets per second written but 300,000 home timeline reads per second, so they fan out on write into Redis timeline caches, roughly 30 average deliveries per tweet. Then there is Katy Perry with over 100 million followers at her peak: one tweet meaning 100 million Redis insertions, minutes of queue time, during which her followers see the tweet at wildly different times and reply chains arrive before the tweet itself. The fix Twitter described in its infrastructure talks is exactly the hybrid: high-follower accounts are excluded from fan-out, and their tweets get merged into timelines at read time. One threshold check at post time routes each tweet down the push path or the pull path.