Loading...
✅ Scale: 100M–1B/day → ~11,600/sec
✅ Latency: <5s
✅ Delivery: at-least-once
✅ Retry: 24hr, exponential backoff, priority
✅ Ordering: for dependent notifications only
✅ Multi-region: data residency + low latency + failover
RedundancyEvery component runs with N+1 or N+2 replicas. No single point of failure.
FailoverAutomatic failover via health checks + leader election (e.g. Zookeeper)
Data replicationAsync replication across regions, synchronous within a region
Queue durabilityMessage queues persisted to disk (Kafka), replicated across brokers
Upstream failuresIf APNS/FCM is down → circuit breaker trips → messages held in retry queue
Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...
PUT /v1/users/{user_id}/preferences
POST /v1/notifications/schedule
POST /v1/notifications
GET /v1/notifications/{notification_id}/status
POST /v1/notifications/batch
POST //v1/engagements/track
Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.
Client sends request → API Gateway validates auth + rate limits
Notification Service enriches the payload (fetches user segments, device tokens)
Pushes to Kafka topic partitioned by priority
Dispatcher Workers consume → fan out to APNS/FCM/WebPush
Each platform responds with delivery receipt
Status written to Analytics DB
Failures → Retry Queue with exponential backoff
notifications.priority.critical → (OTP, security alerts)
notifications.priority.high → (transactional, orders)
notifications.priority.normal → (social, updates)
notifications.priority.low → (marketing, promotions)
Critical → 4 dedicated workers High → 3 dedicated workers Normal → 2 dedicated workers Low → 1 worker (gets remaining capacity)
Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.
Store user's timezone in preferences DB. At dispatch time, convert quiet hours to UTC and compare against current UTC time. If within quiet hours → don't drop it, delay it — push back to Kafka with a scheduled delivery time for when quiet hours end.
Use an idempotency key (notification_id + user_id + device_id) stored in Redis with a TTL. Before processing, check if key exists → if yes, skip. This gives you exactly-once delivery semantics at the worker level.
We will keep a track of notification service dispatcher and we will keep priority basis as well, as I mentioned in the high-level design. And after that, if suppose there is 10 million requests coming to this, we will not be processing them at once. We will be querying the request like 1,000 requests at a time per worker. So for 10 million, we can convert 1,000 workers, processing those 1,000 requests, and in that way, we will be able to handle the service. And anything which is failed will go into a retry queue with the item potency key, and the retry queue will have the same priority, and we can manage them with that. If anything goes bad, like 24 hours, we will go with the exponential backoff. Once everything retry queue is done, we will just put it into the dead letter queue.
We will keep item potency key as user ID, device ID for things, and that's where we can keep the item potency key managed.
We will make sure that we keep the rate limits in check for the APIs and FCMs. We will have a rate limiter for them as well.