Skip to main content
Indexinglesson 4 of 4 · 3 min read

What Writes Pay for Your Reads

Every write pays

An index is a promise your database has to keep on every single write.

Insert one row into a table carrying five indexes and the database does six pieces of work. The row itself, plus an entry in each index, and each of those is a walk down a tree, possibly a page split, all written to the recovery log first. Updates can be worse, because changing an indexed column means removing the old entry and adding a new one.

The rule of thumb you keep hearing holds: a table with ten indexes can write two to five times slower than the same table with two. Storage compounds it, and indexes on a busy table routinely outweigh the table.

The cost hides well. Each of your inserts still looks fast on its own. You find out at bulk-load time, or when replication starts falling behind, or when your log volume doubles overnight.

Indexes need a lifecycle

Give indexes the same lifecycle discipline you give code. Postgres records how often each one is used, and an index with zero uses after a month of real traffic is pure write tax. Drop it.

The accumulation pattern is archaeological, because it is archaeological. Every slow query in the product's history left an index behind, nobody has ever deleted one, and the table now carries eleven.

When the work is genuinely write-heavy, the structure itself changes. LSM-trees, the structure Cassandra and RocksDB use, gather writes in memory and flush them out as sorted files that never change. Scattered index updates become one long sequential write.

Pay for it on the read side, where a lookup may have to consult several files, and in the background work that merges them. B-tree against LSM is the same trade that adding or dropping an index is, reads against writes, settled in a different room.

the shape of it
INSERTevents tablethe row itselfidx_user_idB-tree updateidx_created_atB-tree updateidx_statusB-tree updateWALall of it loggedwrite rowadd entryadd entryadd entryfsync
step 1 of 2
One logical insert becomes four physical structure updates plus log traffic, and every extra index adds another.

Worked example

Diego's events table at an ad-tech company ingests 1,200 rows per second and carries 11 indexes accumulated over three years of dashboard requests. Replication lag starts brushing 30 seconds every evening peak, and bulk backfills that took an hour now take five. He queries pg_stat_user_indexes and finds five indexes with zero scans in six weeks, leftovers from retired features, holding 190 GB between them. Dropping them cuts insert latency from 9 ms to 4 ms and evening replication lag to under 2 seconds. For the quarterly backfill of 800 million rows he goes further: drop two more rebuildable indexes, load the data, recreate them afterward. The load runs in 90 minutes instead of five hours, because building an index once over sorted data is far cheaper than updating it 800 million times.

Indexing: wrapping up

In the real world

  • 01PostgreSQL creates B-tree indexes by default and layers on partial indexes, INCLUDE columns for index-only scans, and CREATE INDEX CONCURRENTLY so production tables can be indexed without blocking writes.
  • 02Uber's 2016 engineering post on moving from Postgres to MySQL centered on index write amplification: Postgres secondary indexes point at physical row locations, so rewriting a row touched every index on it.
  • 03Meta migrated its user database from InnoDB to MyRocks, an LSM-based MySQL engine built on RocksDB, roughly halving storage by trading some B-tree read speed for compressed, write-friendly LSM files.
  • 04Cassandra's storage engine is a pure LSM design, memtables flushed to immutable SSTables with background compaction, which is how it sustains write rates no single B-tree primary could absorb.
  • 05SQLite ships a full B-tree engine and cost-based planner inside nearly every phone and browser, and its EXPLAIN QUERY PLAN output supports the same missing-index detective work at a smaller scale.

Questions people ask

Why is my query not using the index I created?

Usually one of three reasons. The filter is not selective enough, so the planner correctly prefers a sequential scan. The query does not match the index's leftmost columns, so the index cannot serve it. Or an expression blocks it, like WHERE lower(email) = ? when the index is on plain email, which needs an expression index instead. EXPLAIN ANALYZE shows which case you are in.

Should I just index every column to be safe?

No. Each index slows every INSERT, UPDATE, and DELETE on the table and consumes storage and cache space, and an index nobody queries is pure cost. Index the columns that appear in WHERE, JOIN, and ORDER BY clauses of queries you actually run, confirm with EXPLAIN that they get used, and periodically drop the ones usage stats show are dead.

When would I pick an LSM-based store over a B-tree one?

When sustained write volume is the defining constraint, like ingesting events, metrics, or messages at hundreds of thousands of rows per second. LSM-trees turn those writes into sequential disk I/O, which B-trees cannot match. If your workload is read-heavy with moderate writes, which describes most applications, B-tree engines give faster and more predictable lookups.

Quick review

B-Tree:
balanced tree, O(log n) reads, range queries work. Default in PostgreSQL and MySQL InnoDB
Hash index:
O(1) exact-match lookups. Cannot do range queries. Used in hash partitioning
Composite index (a, b, c):
leftmost prefix rule. Index on (a,b,c) serves queries on (a), (a,b), (a,b,c) but NOT (b,c)
Covering index:
index contains all columns the query needs. Row not fetched at all. Index-only scan
Partial index:
only index rows satisfying a condition (WHERE active = true). Smaller, faster, more targeted
LSM-Tree (Log-Structured Merge):
writes to in-memory memtable → flush to SSTables. Fast sequential writes. Used in Cassandra, RocksDB, LevelDB
B-Tree vs LSM-Tree:
B-Tree optimizes reads (fewer I/Os per lookup). LSM-Tree optimizes writes (sequential I/O) at cost of read amplification
Trade-off:
each index speeds reads but slows every INSERT/UPDATE/DELETE. Index only what you query
the trade-off

10 indexes on a table can make writes 2 to 5× slower. Each index needs storage too.

in the room

Add index on every WHERE, JOIN ON, ORDER BY column. Measure with EXPLAIN before and after.