Skip to main content
SQL vs NoSQLlesson 4 of 4 · 3 min read

Polyglot Persistence

Most systems run several

Grown-up systems rarely run one database. The usual shape is a relational store plus a memory cache plus a search engine: truth, hot path, text.

That is fine. What separates a clean version of it from a haunted one is a single rule: one store holds the truth, and everything else holds a copy you could delete and rebuild.

You took something on the moment there were two stores. You now have a synchronising problem, and there is a tempting wrong answer to it.

The tempting wrong answer

The tempting answer is to write twice from your application, first to the database, then to the search engine. Do not ship that. The second write fails sometimes, during a deploy, on a timeout, and the first one does not roll back. The two drift apart in silence, and you find out months later when a seller asks why their product loads fine but never appears in search.

The boring correct answer is different. Your database already writes every change into a log for its own recovery. A tool tails that log and publishes each change onto a stream, and a small consumer applies it to the search engine.

The copy runs milliseconds to seconds behind the truth, and it cannot silently skip a write. If the index gets corrupted you replay the stream and rebuild it. Derived data is a cache with a rebuild button, and the rebuild button is the part you have to actually test before you need it.

Each extra store costs something. Monitoring, upgrades, a new way to fail, another reason to be paged.

And tell people the freshness number before launch rather than after: data appears in search a second after it was written. Two stores is normal, three is common, five is a team that has been saying yes too often.

the shape of it
App serversPostgressource of truthDebeziumtails the WALElasticsearchrebuildable copyRedishot reads, TTLall writescache readschange streamasync upsert
step 1 of 3
Writes go to one place; every other store is fed from the change stream and can be rebuilt from it.

Worked example

Meera's rental-listings site dual-writes: save the listing to Postgres, then index it in Elasticsearch from the same request handler. During a Tuesday deploy the ES client library starts throwing on a connection pool bug, the handler catches and logs the error, and for nine days roughly 3 percent of new listings never reach the index. Nobody notices until a landlord calls asking why his 14 apartments are invisible in search. The audit finds 4,100 missing listings. The rebuild takes a weekend: Debezium tails the Postgres WAL into Kafka, a small consumer upserts into Elasticsearch, and the in-request indexing code is deleted. She then reindexes all 12 million listings from scratch in five hours to flush remaining drift. The next ES outage costs 40 minutes of indexing lag and zero lost documents.

SQL vs NoSQL: wrapping up

In the real world

  • 01Instagram scaled its core data on sharded PostgreSQL, mapping thousands of logical shards onto a smaller set of physical servers so shards could move as the fleet grew.
  • 02Amazon's 2007 Dynamo paper came out of the shopping cart, where refusing writes during failures cost revenue; that lineage became DynamoDB, the managed AP key-value store.
  • 03Netflix runs Cassandra for write-heavy datasets like viewing history, spreading writes across large multi-region clusters instead of scaling one primary vertically.
  • 04Shopify runs sharded MySQL, grouping shops into pods so one merchant's flash sale is isolated from the databases serving everyone else.
  • 05Discord published how it moved message storage from MongoDB to Cassandra and later to ScyllaDB as messages grew into the trillions, while other data stayed in conventional stores.

Questions people ask

Is NoSQL faster than SQL?

Not as a category. A key-value read from Redis beats any SQL query, but a Postgres primary-key lookup takes a millisecond too. NoSQL stores win on horizontal write scaling and on the access patterns they were shaped for, and lose on ad hoc queries and multi-record transactions. Speed comparisons only mean something for one specific query at one specific scale.

Can I just use Postgres for JSON documents instead of MongoDB?

Often yes. Postgres jsonb columns store nested documents, support indexing into them, and you keep transactions and joins for everything else. Teams that want some schema flexibility inside a relational core usually do this. A dedicated document store starts making sense when nearly all data is document-shaped and write volume needs horizontal scaling.

How many databases is too many?

Count operational burdens, not technologies. Each store needs backups, monitoring, upgrades, and someone who understands its failure modes at 3am. Two or three with clear roles (truth, cache, search) is a normal production setup. If you cannot say in one sentence which store owns the truth for a given piece of data, that is the real problem.

Quick review

Relational (SQL):
fixed schema, ACID, arbitrary JOINs. PostgreSQL, MySQL. Best for complex queries and transactions
Document:
JSON/BSON, flexible schema, nested data. MongoDB, DynamoDB. Best for hierarchical/semi-structured data
Key-Value:
simplest model, O(1) get/set. Redis, DynamoDB, Memcached. Best for caching and session storage
Wide-Column:
row key + dynamic column families. Cassandra (AP), HBase (CP). Best for write-heavy time-series, IoT
Graph:
nodes + edges + properties. Neo4j, Amazon Neptune. Best for social graphs, recommendation engines, fraud detection
Time Series:
append-only timestamped data. InfluxDB, TimescaleDB. Best for metrics, telemetry, financial ticks
Search:
inverted index, full-text ranking. Elasticsearch, Solr. Best for product search, log search
Capacity anchors:
single-node RDBMS ~ thousands of writes/sec; Cassandra and DynamoDB scale writes near-linearly by adding nodes/partitions
Rule:
pick based on your dominant query pattern. Many production systems use 2 to 3 database types (e.g., Postgres + Redis + Elasticsearch)
the trade-off

NoSQL buys horizontal scale and schema freedom by giving up joins, ad-hoc queries, and multi-row transactions, and that work moves into your application code. Choose from the queries you already know you need, because access patterns get baked into the key design and changing them later means a migration.

in the room

Default to PostgreSQL for most apps. Add specialist databases only when PostgreSQL can't meet a specific access pattern.