Loading...
List functional requirements for the system (Ask the chat bot for hints if stuck.)...
List non-functional requirements for the system...
Estimate the scale of the system you are going to design...
Define what APIs are expected from the system...
1 - We will use web sockets for the main messaging functionality as it is bidirectional communication. For example if a user is active in the chat session, then we can push messages to them in real time without them needing to poll.
2 - load_chat_history. when users open the app and open a chat, we will load the entire conversation history we have
{conversation_history: [{sender: message, timestamp, receiver:message},...]}
we will use a standard HTTP call here. tried and true, no need to get fancy
3 - create_chat_session {user_id_1, user_id_2}
return a chat_session_id
3 - join_session{chat_session_id: }
this will tell the server that a user is active in a certain chat
4- leave_session{chat_session_id: }
when a user leaves the chat session
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
ChatMessages
This is massive, we will get 1TB of new data every day. Given the massive scale of 14M QPS, we will have heavy read and write pressure. Since the schema should be quite simple and there are no complex relationships we should use a key value db like DynamoDB for this use case.
{chat_session_id, user_id, message, timestamp} are the key fields.
chat_session_id should be the partition key, and timestamp can be the sort key. We should logically group all messages in the same chat together in the same partition.
User Profiles
This is not as important to the core design, but will just mention that when users register and add profile information, we can store this here. There will be load read and write pressure, and schema should not change much. We can use a relational db like MySQL to hold information like this.
Sessions Store
We will store in cache users active sessions. it will be keyed by chat session id, and contain a list of active users in the session, and also the chat web server that they should connect to. We choose a cache here because sessions are short lived, simple schema, and super fast reads. We can quickly evict as well.
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
Client makes calls to service first hits our load balancer. For chatting it will connect with websocket. Our load balancer needs to maintain a sticky session with the compute since if both users are active in the session at the same time, we will be pushing messages to the receive with websockets.
Web server must write to our data layer to save the messages. During load chat history, it will also need to read from there.
Web server also checks the redis active sessions cache to route users to any relevant chat web servers.
Chat web servers will only handle the sending messages and websockets part.
Notification queue and service will handle sending notifications if user is not active in the chat session.
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
User joins a session, service checks the session cache for whether it is currently active and has a connection to a web server. Connect the user to the active chat web server if entry is found. update the cache with the user_id to signify the user is active.
user sends message to chat web server. This web server will hold temporary state in memory. if their counterpart is also currently online, it will directly push the message to that open connection with the chat counterpart.
if not, it will queue up a message to the notification service queue. notification service will pick it up and send the notification
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
Lets dig into the active chat sessions business.
The redis store should have schema like {chat_session_id, active_users, web server id}
When a user joins session, we will check whether the session_id is in the cache. If it is not, create the entry, pick a chat web server (least used?) and also place the user_id into the active_users list.
Say their friend joins, we will check the cache again, the chat_session will exist, and we will add their user id into the list, and also connect the second friend to the same chat web server.
When a user initially sends a message to the chat web server, we will also keep an in memory store on the server, that is keyed by chat session id. it will hold any active connections to the users currently. So whenever it receives a websocket message request, it will first check in memory if there is an active connection for that chat session (thats not the sender), and push the message. If that does not exist, then it will push the message to the notification queue.
We can also get some easy wins by adding a redis cache in front of the ddb chat service. I think this will put a lot of read pressure off of DDB. it can store entire chat history sessions, and if a user is constantly closing and oppening their app, then chat history sessions can be cached. we should use cache aside strategy and configure a LRU eviction policy. This is a straight forward way that works well for when a user is actively using the app a lot but exiting and re entering the chat, which is common with users.
Explain any trade offs you have made and why you made certain tech choices...
Adding a cache for chat history collection might be considered overkill, it could complicate the app logic, but should relieve some read pressure.
Another choice was to use NoSQL for chat messages. This is a good choice since we need efficient writes for a simple schema, massively scalable data. One tradeoff is DDB is not strongly consistent by default, which is concerning and fails to meet one of my non-functional requirements. DDB can be configured to be strongly consistent but costs more.
We are also storing sessions on the chat web servers itself, which breaks servless architecture, but in the case of short lived websocket sessions, this should be an okay choice.
Try to discuss as many failure scenarios/bottlenecks as possible.
If a chat web server fails, then both users will be kicked out of their session. In this case, we need to clean up their active entry in the Redis cache. IN the most simple case we will just require the users to refresh their chat with a call to get chat history for consistency, and when they submit their new chat, we can connect them to a new server.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?
We should look into end to end encryption.
we should look into how to gracefully move users connections to a new chat server if theirs dies.