Loading...
Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...
For this flow, we need the below APIs:
An API used to establish a connection through web socket, this should happen at suers signup for the service or introducing of a new device for a user:
POST /v1/establish_connection {
user_id: UUID,
device_id: UUID,
sign_up_at: Timestamp,
}
An API used to retrieve a friend's profile:
GET/v1/retrieve_user_profile {
user_id: UUID
}
An API used to query the online/offline status of all friends
GET /v1/query_friends_status {
user_id: UUID
}
This would do a batch presense lookup for the current registered friends of the user.
An API that allows user to update their online status
POST /v1/update_status {
user_id: UUID,
status: String,
update_time: Timestamp
}
An API that allows user to subscribe to the updates of a friend's presence changes:
GET /v1/register_friend {
user_id: UUID,
register_friend_id: UUID,
register_type: string
}
An API that allows the client to periodically send updates to the server to prove liveness
GET /V1/check_liveness {
user_id: UUID,
}
Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.
Let me walk through the several major workflows.
To maintain persistent connections between clients and the server, we will use websockets which maintain bi-directional connections between the client and our service.In the user presence coordination server, we consolidate the status for all the clients of a user.
Each user has a list of friends who they update their online status to. We have a relational database that stores the user and their friends.
table friends {
user_id: UUID,
friend_user_id: UUID,
register_presence_update: boolean
}
Whenever uses add/register a new friend, we go through the flow to update the redis cluster and relational db. It will be a write through cache, with higher write latency, but strong consistency between cache and the db. This is acceptable since we only update the friends relational db when users add or register/unregister friends. We store the mapping between each user and all of their associated friends.
Within redis, we store:
key: user_id
value: {
registered_friends_user_ids:
registered_friends_online_status:
}
Whenever a user becomes online, the client will update the user presence coordinator server through websocket. We then pull the list of registered friends for that user from redis, and use web socket to send notifications to the friend's devices. For each of that user's friend, we update the registered_friends_online_status in redis.
When a user want to query the online status of friends, we simply query redis and can see the output.
Whenever a user registers a friend, adds a new client, or goes online/offline, we generate a kafka event at a designated topic. On the consumer side, we process the event, and add a log to a cassandra database. Cassandra is good here since it can handle huge amount of write traffic, and is highly available. The consistency tradeoff here is acceptable since we don't need logs to be immediately consistent.
Sequence of events in the online update path:
For flapping connections, we can add a grace period during which the system allows a user to remain online if they experience brief disconnections. For example, if a user disconnects, we hold for 30 seconds before triggering the "user becomes offline" flow. If they become online within 30 seconds, nothing changes.
For handling multiple devices of a user, we maintain a separate connection for each device of the user. As long as any of the device comes online, we trigger the online event. The user presence coordinator server coordinates events received from multiple clients of a user/
Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.
Let's do a deep dive on analytics.
To ensure our system has good observability and analytics, we should create:
We should also add logs to all of the users' status updates, including adding/removing/registering friends, going online/offline etc. We will apply log sampling to keep logs that have errors, as well as 10% of regular logs. For logs, we will create a kafka topic that receives events for user online status updates. Upon processing the events, we write to cassandra database. Cassandra is good for storing event logs and doing analysis, since it has high write throughput, is highly scalabale. It is eventually consistent, which is okay for our use case.
For the metrics we build, we will write the updates to a metric aggregation service like prometheus through the designated APIs.
Let's also talk about a heartbeat manager.
Sometimes a client could have become inactive, and didn't explicitly disconnect the connections. In this case, we can build a heartbeat manager, that sends periodic heartbeat checks to clients, and detect inactive clients.
This can give us a more accurate update in case of user's status changes. Also, the heartbeat manager can differentiate between a momentary lapse in connectivity, or a genuine disruption for a client.
For each client, we will have a TTL of 30 seconds, with a user considered offline if no heartbeat is received from this user within this time. We will use a sliding window to look at this, where each heartbeat resets the TTL.
For the fanout of a user's status to all of their friends. We can shard the redis cluster if the friends relations are too complex and vast. A good partition key is user_id. This allows us to more quickly find the registered friends of a user, and their online/offline statuses. We can batch the update requests, and instead of sending update request through websocket to individual friend, we send batch requests to all registered friends, reducing the network burden.