Skip to main content
Storage & Throughputlesson 1 of 3 · 3 min read

Sizing Data Honestly

Start from one record

Every storage estimate starts with the size of one record, and this is where most estimates die.

Guessing a few KB is where it dies. Build the number yourself. A tweet caps at 280 characters, so the text is about 280 bytes. Add a user identifier, timestamps, references to media and a few denormalised counters, and you are near a kilobyte as stored. A compressed photo runs around 300 KB. An average web page weighs about 2 MB with its assets. A minute of high definition video is roughly 50 MB. Those four anchors cover most consumer products.

The multipliers nobody puts on the slide

Then apply the multipliers that turn raw sizes into real disk usage. Your indexes commonly add 30 to 100 percent on top of the table data. Replication multiplies everything by your copy count, usually three. Your backups, logs and snapshots pile on after that.

One workable rule covers it. Double the raw number for indexes and overhead, then triple it for replication, and treat that as your honest footprint.

One worked end to end. A service ingesting 500 million posts a day at a kilobyte each stores 500 GB a day, roughly 180 TB a year raw and 550 TB with three copies. Now add media. If one post in ten carries a 300 KB photo, that is 50 million photos a day, 15 TB a day, about thirty times the text.

The lesson in that gap: media dwarfs metadata. It is why photos go to object storage behind a content delivery network while the text sits in a database.

Units are worth keeping straight with powers of ten. A kilobyte is ten to the third bytes, a gigabyte ten to the ninth, a terabyte ten to the twelfth, a petabyte ten to the fifteenth. Interviews use decimal units because the arithmetic stays clean.

Every assumption belongs out loud. An interviewer will happily accept “assume a kilobyte per record” and will absolutely punish a silent guess buried in your total.

the shape of it
Message text200 bytes+ metadatasender, time, state+ indexesoften 2-3x+ replicas, backups2 TB, not 150 GB
step 1 of 3
The number people quote is the first box. The disk you buy is the last one.

Worked example

Dana is sizing storage for a Slack-style chat product pitched at enterprise customers. Instead of guessing, she builds a message: 100 bytes of text on average, sender ID, channel ID, timestamp, and the overhead of JSON, the text format the records are stored in, lands near 300 bytes stored; she rounds up to 500 to be safe. Ten million daily active users sending 30 messages each is 300 million messages a day, times 500 bytes is 150 GB a day, about 55 TB a year. Tripled for replication: 165 TB. Then she checks attachments: if 2 percent of messages carry a 300 KB file, that's 6 million files a day, 1.8 TB daily, about 650 TB a year, four times the message text. Her one-slide conclusion: messages fit a database cluster comfortably, and the file store is the real bill, so it goes to object storage with lifecycle rules.