Why not guarantee exactly-once delivery?

Because the last hop makes it impossible: when a call to APNs or Twilio times out, you cannot know whether the message went out, so any retry risks a duplicate and any non-retry risks a drop. Systems therefore promise at-least-once and invest in dedup, idempotency keys at the API, sent-markers before gateway calls, and collapse keys on the device, so the rare duplicate is harmless.

How should priorities be implemented, one queue with priority fields or separate queues?

Separate queues (or Kafka topics) per priority tier, each with its own worker pool. A priority field in a single queue does not help once a 20-million-message campaign is physically ahead of your password reset in the partition. Separate lanes give transactional traffic dedicated drain capacity that marketing bursts cannot consume.

Where do user preferences get enforced?

In the notification service, on every send, against the live preference store. Producer teams should not even be able to express send this regardless of preferences, except for a narrow security category. Centralizing the check is what makes compliance auditable: one code path, one log, one place to prove an opted-out user was never contacted.

Common Designschapter 6 of 8 · 4 lessons

Notification System

Reliably deliver push, email, and SMS notifications at millions per second.

← All chapters View in cheatsheet

Marketing schedules a campaign to 20 million people for 9am. At 09:00:01 it lands in your notification system all at once.

Somewhere behind those 20 million messages is a password reset code, for somebody staring at a login screen right now. If both go through the same queue, that code arrives eleven minutes later, which is to say it may as well not arrive at all.

Now add that some of these cost money to send, that people can opt out of some categories and not others by law, and that a phone number can be wrong forever while an email address is fine.

It is the post office of a backend. Every other service hands it messages, and it worries about addresses, delivery, return to sender, and how much any of it costs.

Lessons

4 in this chapter

Requirements and the API ContractYou are building a platform other teams call, so the API contract is a first-class design decision.3 min
The Delivery PipelineEvent in, Kafka between every stage, channel workers out, and third parties at the edge.3 min
Retries, Dead Letters, and IdempotencyAt-least-once delivery plus idempotency keys beats chasing exactly-once across third parties.3 min
Preferences, Rate Limits, and PriorityThe features that protect users from your own company are the ones that keep notifications working.3 min