Skip to main content
News Feed / Timelinelesson 1 of 4 · 2 min read

Requirements and Estimation

Scope and shape

Settle the functional scope first. People follow other people, post text and pictures, and open a feed of recent posts from the accounts they follow, roughly newest first.

Ask whether that feed is chronological or ranked, because ranked adds a whole machine learning layer underneath.

Ask about the shape of the follow graph too. Mutual friendships cap your fan-out naturally, while one-way follows allow accounts with a hundred million followers, and those accounts will dominate your design.

Take numbers for a large product: a hundred million daily users. Posting is rare and reading is constant. Say one in ten posts twice a day, giving 20 million posts, about 230 a second.

Watch feed opens dwarf that. Each active person checks five times a day, so 500 million loads, roughly 5,800 a second on average and 20,000 at peak. Your raw traffic already says read-optimised, so precompute whatever you can.

The multiplier

Now find the multiplier that makes feeds interesting. If your average poster has 200 followers and you push each post into every one of their feeds, 230 posts a second becomes 46,000 insertions a second.

Recognise what you just did: turned a modest write load into a heavy one so that reads become instant.

Consider the alternative, assembling feeds when they are read, which turns each of your 20,000 peak reads into queries against hundreds of accounts.

Neither pure strategy survives your follower distribution, because it is a power law. A typical person has 200 followers and the tail accounts have 50 million.

Set your targets: feeds render in a couple of hundred milliseconds, and freshness is loose, because a post appearing 30 seconds late is invisible. That gap, strict latency and soft freshness, is exactly where precomputation lives.

the shape of it
200 followers50M followersPush on write200 writesPush on write50M writesfinenever finishes
One strategy, two wildly different bills, because follower counts follow a power law.

Worked example

Rohan is designing the feed for a fitness app where users follow trainers. He assumes the follow graph looks like Twitter's and designs pure fan-out on write. Then he pulls the actual distribution from the database: 40,000 trainers, and the top one, a yoga instructor named Karina, has 2.1 million followers out of 3 million total users. When Karina posts her morning routine at 6 am, fan-out on write means 2.1 million feed insertions for one post, and she posts daily. Meanwhile the median trainer has 90 followers. His revised math: fan-out on write for the 39,950 normal trainers costs a trivial 100 insertions per post, and the 50 accounts over 100,000 followers get pulled at read time instead. Two strategies, chosen per account by one number in a config table.