Requirements

Functional Requirements:

• Send targeted notifications based on user attributes, device, and preferences.
• Schedule notification delivery for immediate or delayed send.
• Deliver notifications across multiple platforms including iOS (APNS), Android (FCM), and web browsers (Web Push).
• Track user engagement with notifications including clicks, dismissals, and opens.
• Manage user opt-in/opt-out preferences and notification settings.

Non-Functional Requirements:

• reliability - the notifications should always be delivered, even if there is a delay or they arrive unordered
• scalability - the system should be able to scale out during high peaks, also it should be able to deliver millions of notifications each day
• no data loss should happen, the notifications can be delayed but they almost must be delivered
• no single point of failure should exist, we should achieve this by using message queues
• high availability: should be up 99.99% of the time and all the notifications should eventually be delivered
• fault tollerance should be taken into account and fixed by implementing retrial mechanisms, like leveraging message queues and pushing failed messages back to the queues to be reprocessed

API Design

POST /notifications/preferences/{userId} body:{notificationType: string; isEnabled:boolean} - with this endpoint, the users can turn on/off specific notifications receival

GET /analytics/notifications for admins to track user engagements with notifications

POST /analytics/action/user/{userId}/notification/{notificationId} - used when user clicks on notifications, in order to saved the action to db

GET /analytics/notifications/{userId} for admins to track a user engagements with notifications (clicks,dismisses)

GET /notifications/preferences/{userId} - gets all notification preferences for a specific user if provided or for all users

GET /notifications/{userId} - gets all received notifications for a user or all users - used by admins

POST /send/notification body:{userId:string; message:string; type:SMS | mail | APNS | FCM} used to create the notification entry on our side and calls the third party provider for sending the notification

GET /notifications/{notificationId} - gets the info for a specific notification (status, sentAt, userId, type) about its delivery status, when was it sent, etc

POST /send/notifications body:{userIds:JSON; message:string; type:SMS | mail | APNS | FCM} used to create the notification entries on our side and calls the third party provider for sending the same notification to multiple users at once - eg for updates or global messages

High-Level Design

updating user notification preferences flow:

1. client makes api call which reaches load balancer
2. load balancer routes call to a notificaton preferences service
3. service updates notification preferences DB

sending notifications flow:

1. the service calls notifications servers
2. notification servers handles authentication and rate limiting; they call the cache to check users' preferences and device info. in case of missing cache, they call notification preferences DB to see if the user disabled that notification and then the device info db to fetch the needed details to send notifications (tokens, deviceId, userId, etc). also here a notification entry is created in the db with status in_progress and analytics service is also called to create the entry. also here we check for duplicates: in case the notification entry already exists in the DB then we stop the current flow, meaning a different thread is already managing this notification
3. the notification is then pushed to the corresponding queue: iOS PN queue for iOS devices push notifications, FCM PN for android and WEB PN for web.
4. workers then consume from queues and in case of errors they retry delivery by repushing the messages to the queue. also they fetch from notification templates and construct the entire message with the missing variables. they update the analytics service and mark the notifications as delivered.
5. workers then call the corresponding APIs, APN for iOS, FCM for Android, and Web Push API for web. these also update the device info db in case the tokens are expired and then push the notifications to devices

scheduled notifications flow:

1. service calls notifications servers, they add a scheduledDeliveryDate:date to notifications and do the same as the above flow, except for pushing to queues; instead they push it to scheduled jobs DB
2. cron service keeps polling on scheduled jobs and when the target time is reached, it pushed the messages to the queues and continues the flow

Detailed Component Design

service calls notification servers, which can be scaled out during high spikes automatically using kubernetes (it increases the number of pods when in need);

then based on notification type (iOS, android, web) it is mapped to the corresponding queue (iOS PN queue, FCM PN queue or web queue), then each queue gets consumed from by workers. and then the workers call the corresponding delivery service with the notification message: APN, FCM and web push api which push notifications to devices.

in case of failures, notification sending is being retried by pushing the message back to queues to be reprocessed. no notification is sent twice because it is saved in the db with its status and in case a second thread also tries to deliver it, it firstly checks the db to make sure the notificaton isnt already being processed.

during high traffic notifications cant be lost because they are pushed to queues, we can increase the number fo queues if they reach 90% capacity; and workers consume from them at their own pace, so even if the notifications are delivered with a delay, they are surely delivered.