Why WebSocket instead of polling or server-sent events?

Polling adds latency (up to the poll interval) and wastes battery and server capacity on empty responses. Server-sent events are one-directional, server to client, so sends still need separate HTTP requests. WebSocket gives one bidirectional connection for both directions, which is why every major chat product uses it or an equivalent persistent socket protocol.

What happens to a message if the recipient's chat server crashes?

Nothing is lost, because the message was persisted before the publish. The recipient's client detects the dead socket, reconnects through the load balancer to a healthy server, and syncs from its per-conversation cursor, picking up anything published while it was disconnected. This is why persist-then-publish ordering and cursor-based sync are the backbone of reliability, not the pub/sub layer.

How do you keep messages in order?

Guarantee order per conversation only, using a time-sortable server-assigned message ID as the single source of truth. Clients render by ID order, and any message arriving out of order slots into place. Global ordering across conversations is unnecessary and would require coordination that kills throughput.

Common Designschapter 2 of 8 · 4 lessons

Chat System (WhatsApp / Slack)

Real-time bidirectional messaging with offline delivery and group support.

← All chapters View in cheatsheet

You send a message and it appears on your friend's phone before you have put yours down. Two ticks, then two blue ticks.

Everything else on the web waits to be asked. Your browser requests a page, the server answers, the connection closes. Here the server has to reach a phone it is not currently being asked anything by, and do it in under a few hundred milliseconds, ten million times over.

Run the numbers on that last part. If a fifth of fifty million daily users are online at the evening peak, that is ten million connections held open at once, spread over a fleet, and something has to know which machine holds which person.

Connection state is what makes chat harder than any ordinary API, and almost every decision in this chapter comes back to it.

Lessons

4 in this chapter

Requirements and EstimationMessages per second is easy; millions of open connections is the real number.2 min
Message Flow Across ServersTwo users on two different servers is the problem; a pub/sub backbone is the answer.2 min
Storage and Offline DeliveryChat history is an append-only time series, and offline sync is just a cursor into it.3 min
Groups, Receipts, and PresenceThe features users take for granted are where the write amplification hides.2 min