Loading...
List functional requirements for the system (Ask the chat bot for hints if stuck.)...
List non-functional requirements for the system...
Estimate the scale of the system you are going to design...
Define what APIs are expected from the system...
We will not use REST API, will opt for websocket instead. Because it's a real-time chat app that requires two-way communication, if we consider Rest API and send a POST and GET request every time the user wants to send and receive a message, it will create a lot of overload on the servers.
payload:
{
from: user1,
to: user2,
content: xxxx,
timestamp: xxx,
}
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
User DB:
We need a user database to store user info including user_id, name, email..etc.
Metadata DB:
We need a time series database as metadata database to store content of the messages, the structure will be id, timestamp, format (enum: text, picture, video), text_content, link (if it is picture/video, we need to store them in a BLOB storage and link it to the metadata DB), read/unread. Opting for time series database because for chat messages, there will be a lot of reads and writes, and they should be displayed by the timestamp and time series database offers us a easy way to fetch messages over time period as well as providing support for high volume of reads and writes with low latency. On top of that, we don't need to consider transaction as it is chat and it's ok if we see duplicate messages once in a while, meaning relational database isn't necessary here.
BLOB Storage:
We can use BLOB storage such as Amazon S3 to store the media files linked in chat
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
flowchart TD
B[client] --> C{load balancer}
F[user database]-->D
C-->D{server}
D --> E[metadata DB]
G[BLOB]-->D
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
user1 is in server1, user1 sends a message to server1, the message is appended to a message queue. Server1 processes the next message in the queue, sees that user 1 is trying to send a message to user 2, server1 checks with zookeeper service to find the server user2 is connected to, then sends to message to that server. Server 1 also stores the next message in the queue to storage, if there is media file, server 1 store the file to S3 (BLOB), gets the link to file, then stores the message to metadata DB with the link to file; Otherwise, server 1 stores message to metadata DB directly.
User 2 periodically sends heartbeat to server 2 if they are online, when server 2 consumes the next message in message queues and sees a message for user 2, server 2 checks if the heartbeat timestamps from user 2 are expired, if all are expired, user 2 is not online and server 2 needs to push notification for user 2 notifying there is a new message. Otherwise, user 2 is online and once user 2 has seen the message, the message is processed and server 2 updates the status of the message to read in the metadata DB.
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
Explain any trade offs you have made and why you made certain tech choices...
Try to discuss as many failure scenarios/bottlenecks as possible.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?