System requirements

Functional:

• Real Time Presence Indication
• Trigger Notification when a user becomes online.
• Allow Notification to be opt-in
• After X mins of Inactivity, mark User InActive
• One or more users can see the statuses

Non-Functional:

• Highly Available
• Scalable to cater growing demand of users

Capacity estimation

• 10 million MAU
• Assuming 10% of MAU is PEak concurrent user, 1 Million (10% of MAU) concurrent users at PEAK Times
• averaging 10 times the status update, it will be 10Million updates at peak, 120K QPS at peak for 1 to one user.
• Now, if a user lets say max friends/connections is 500, then peak QPS is 600M QPS
• Assuming per user data is 10MB, 100TB data per month, 10TB at peak, 1.2PB per year, for 10 years we will need 12PB, with buffer, lets say we would need 20PB to scale better

API design

User's presence is described as Online and Offline

• GET - /api/v1/presence/status
• POST - /api/v1/presence/status

For Each User in connections we expose a notify API, if the connections subscribe to notify (Yes). they are notified each time the user status changes Online -> Offline and Offline -> Online

• POST - /api/v1/presence/notify

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...

High-level design

Request flows

• User Logs in to the Application
• Application (main) sets up a WebSocket connection and calls presence Service
• Presence Service sets UserStatus to Online and ActivityStatus to Active immediately while also setting LastUpdateTime.
• EachTime Users makes a request to the Application, Presence Service POST API `/api/v1/presence/status` is called which sets user status to Online
• In The Background we run a Task, that would run every minute and read the user presence status, if status is Online and last_updated_status is greater than 5 mins. User is marked Offline
• Presence Service on each status change publishes to Message Queue.
• All Connections that Opt-in for the notification on their(connections) are subscribed to the MessageQueue

Detailed component design

• WebSocket Connection: Our Application opens a WebSocket for each session. This means the evens happens at the real time
• Columnar DB: Our system is both Read and Write Heavy. We also need to store 12 PB of data. RDBMS won't scale. Also, to efficiently set user statuses we we will read last_update column and user. Columnar DB would make our queries very fast as data is stored by columns.
• Message Queue- Each Users gets a message to manage Notification for their Activity. Also, each user can susbcribe to the Queues as many as the connections they may have. With Each Notification change for a User an event is published to the message queue.
• Worker Pattern to Manage User Activity. Presence Service start a Task that, every 5 mins checks the User Status if the status is not changed since, it marks user as Offline. This is done by using worker pattern. If there is a task failure worker, task manager schedules that task on another worker

Trade offs/Tech choices

Failure scenarios/bottlenecks

• We need to handle race condition. For instance whet happens when presence service write to user status and at the same time background job runs. We can't allow last write wins here, because if the last write is background job it may incorrectly set the status when presence service would have marked user as online. We also can't use DB locks as we may run into too many locks as we have about 600M QPS and Background task of even 10% would be 60M QPS. So, we take an alternate strategy where in, we grant lease, when a request is processed it acquires the lease and when it finishes the releases the lease.
• We also need to handle long await notification errors. Consider what happens when user don't have access to network for an hour, do we notify this user of all activity when they come online or we clean up the queue and starts fresh. We trade efficiency over correctness, so we will clean up queue and start fresh. so user is not floodded with notifications and our service can be widely adopted in the range of applications.

Failures and handling

• We implement Retry Strategy Pattern with Exponential Backoff and Jitter
• KPI monitoring for
  • Queue Size (to ensure latency)
  • DB latency
  • Network bandwidth
  • CPUs/IO and Disk Stats
  • Service UpTime

Future improvements