System requirements

Functional:

List functional requirements for the system (Ask the chat bot for hints if stuck.)...

• detect if user is online or offline
  • online without activity for certain period of time = idle
• allow retrieval of current user status
• publish or push user status events for real-time updates

Non-Functional:

List non-functional requirements for the system...

• does not need to be highly available -- low user visibility to temporary failure (95% uptime)
• does not need to be highly reliable -- low user visibility to data loss + frequent data updates
• DOES need to be highly scalable -- will be receiving traffic and making data updates frequently

Capacity estimation

Estimate the scale of the system you are going to design...

In designing an Online Presence Indicator Service for a platform like Facebook or LinkedIn, we can plan for a scale similar to these services. Let me provide you with some concrete estimations:

• Total Users: Up to 1 billion users (such as Facebook's scale).
• Monthly Active Users: Around 500 million.
• Concurrent Online Users: Approximately 100 million at peak times.
• Average Devices Per User: 1.5 devices per user, leading to about 150 million simultaneous connections.

the traffic to this service will be based on user activity. if we assume 100 million concurrent users, we can expect 100 million requests over some period of time (we can choose 100 million requests / minute)

there will also be high throughput in this service. the service should check on a regular basis if all online users are still active. assuming 100 million active users at a time, throughput will be 100 million * the size of an activity checking message. assuming a pretty small message (~100 chars), we can estimate 400 bytes * 100 million = 40 GB / minute.

our db will hold the status of all 1 billion users. assuming a pretty small row per user (~100 chars), we can estimate 400 bytes * 1 billion = 400 GB of space.

finally, we will need a pub/sub or websocket that will be receiving messages on change. assuming 5% of users change status per minute, this will be 100 million * .05 = 5 million messages / minute (2.5 GB / minute)

API design

Define what APIs are expected from the system...

update user status:

PUT /user//status

request body:

• status

response body:

• userId
• status
• createdOn
• updatedOn

get user status:

GET /user//status

response body:

• userId
• status
• createdOn
• updatedOn

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...

this system calls for the use of a relational database. we do not need a very highly available service, and the data model will be incredibly simple: we just need to map a user id to a status.

the data model will look like the following:

table: user_status

columns:

• user_id UUID
• status varchar
• createdOn datetime
• updatedOn datetime

High-level design

You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...

This design is relatively write-heavy. The system will continuously be receiving client-side feedback on whether a user is active and writing to the database and to a pub/sub or websocket on change in user activity. On the other hand, the system should only need to serve a read request when a user opens the platform on a new device (because the device should otherwise be receiving status information via the pub/sub or websocket).

To support this architecture, we will use the following:

• A load balancer which distributes requests to one of multiple server replicas
• A server-side cache which provides user status if present
• A database partitioned by user id
• A cron job to update user statuses to idle asynchronously
• A pub/sub queue or websocket to push user status updates
  • Queue can also be partitioned on user id

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

update user status:

• update user status endpoint is called by the client in the event of user activity: user is logged in + takes some action
• load balancer distributes request to a server replica
• server retrieves user status from partitioned database, or retrieves from cache if the status is cached
• if the user status is different from the persisted status, perform the following:
  • update the cache
  • update the database row
  • publish the update event to the queue or websocket

get user status:

• get user status endpoint is called by the client when a new user logs in and needs to view statuses of their connections for the first time
• load balancer distributes request to a server replica
• server retrieves user status from partitioned database, or retrieves from cache if the status is cached
• return the status + update the cache

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

idle status job: this job can be some kind of cron job that runs more frequently than the idle + offline timeouts. for example, if a user becomes idle after 5 minutes of inactivity and becomes offline after 20 minutes of inactivity, the cron job can run once a minute. the purpose of this job will be to pick up any users marked as "active" or "idle" in the database and check their updatedOn timestamp. if this timestamp is too far in the past, the job will mark the user as "idle" or "offline" and publish a message.

this job will have to scale according to throughput -- i.e. how many users are marked as active or idle at one time? this job can easily scale horizontally by partitioning on user id. the effects of the job are also idempotent, meaning there is no consequence to occasionally processing the same user multiple times.

pub/sub or websocket: the purpose of this queue or websocket will be to inform clients when there is a change to a user status. when a user logs in on a device, that client will subscribe to the pub/sub or open the websocket and begin receiving updates re: that user's connections, which the client can then update for the user to see in semi-realtime.

this approach could quickly become very noisy when each client may only care about a small subset of the updates. can we optimize by allowing the client to configure what updates they receive?

cache: this cache will become out of date pretty quickly, when user statuses become idle or offline in a matter of minutes. we can use an asynchronous event queue to achieve eventual consistency in the cache. there is not a high consequence to marking a user as idle or offline multiple times.

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

• I chose to optimize the database by partitioning rather than adding read replicas for a couple reasons:
  • this is a write-heavy system, meaning that it will benefit from having multiple different database instances that are allowed to receive writes
  • this is a write-heavy system, meaning read replicas could quickly become out of date
• I chose to have a server-side cache because an individual user is unlikely to load the same data more than once, meaning browser cacheing is not very relevant. the data will most often be used as a part of the business logic, meaning CDN located close to the user is not very useful

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

• the idle status job will need to scale with throughput. as active/idle users increase, this job may begin processing more slowly than it's scheduled to run without proper resourcing
• the idle status job always has the possibility of being run multiple times on the same user
• the cache can become out of date

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?

• add a bulk "get user statuses by ids" endpoint to allow for retrieving a user's entire connection list at once
• partition idle status job by user id
• keep cache up-to-date with a refresh queue