Loading...
Create a real time messaging app which allows users to send messages, receive messages, shows a tick when a message is sent and a double tick when a message is received.
Latency and availability are both very important for the system. We want the messages to be sent and received quickly. We also want to make sure no messages are lost in transit and that if a user's device does not have signal, that they can receive the message when they do.
Assuming 1 billion active users, there's probably 100 requests at least per user monthly. So that is 100 billion requests a month, 330000000 million a day and therefore 330000 requests a day or 330 a second. Assuming an average message length of 100 50 characters, thats 50 bytes a message. Plus metadata lets say 1kb bytes. that is 1000 byte * 100 billion is about 100 trillion bytes or 100 terabytes of data monthly. Let's assume we are not sending any attachments or videos in this scenerio for simplicity. Lets say we also want to store this information for a year. That is 100 terabytes of data a month a 1200 terabytes a year.
https://facebook/messages/:chat_id which will get a GET request pulling that chat messages. Returning a list of chat messages with meta data on 200 response. Otherwise a 404 response for error.
{
'message_ID': "",
'created_at: ,
'message': ""
'user_id: ""
}
Within the chat we will have a websocket connection we will have:
join_chat(user_id, chat_id) and sending messages which returns a 200 success or 404 error.
{
'message_ID': "",
'created_at: ,
'message': ""
'user_id: ""
}
also we have create_chat(user_id[]) which returns a { chat_id } status 201, or 404 on error.
We are using a NoSQL database for our chats given that we will be ready heavy on first time load of the chat and want to reduce latency.
{
'message_ID': PRIMARY KEY,
'timestamp: ,
'message': ""
'user_id: "",
'profile_image': ''
created_at: "",
location: "",
}
We also include a redis cache between our chat server and our nosql database. This is to store the recent conversations etc so that for first time load the chat is quicker. We can use chat_id as the unique key here.
Our user server uses a sql database for acid compliance, to make sure our data is consistent even if latency is increased here.
The user schema will be
{
user_name,
profile_image,
first_name,
last_name,
created_at,
}
We use an api gateway to support our microservice architecture. this will include a built in rate-limiter, authentication validating client JWT tokens, and distributing the load to the correct servers, weather its user or chat server.
We then have a load balancer that will distribute the load based on weighted round robin.
From chat server we store the most recent chats based on chatID in our Redis cache and from there to our no sql database. This will give us faster load times on first time load of the chat. We will also need to make a request to the user server on first time chat load, so that our profile information etc will also load for us.
Our nosql database will be sharded based on geographical location in which the chat was stared, as most chats will probably be within the same geographical region.
The user server is used for storing user signups and information in our sql database which will be acid compliant and have data consistency even as we horizontally scale.
Client sends a request that is picked up by api gateway.
Api gateway redirects this to our load balancer which then redirects to the correct server based on load. If the request was message related it will store the information in the chat server, otherwise if it we are storing user info it will direct to user server.
Our chat server creates a websocket broker per chat_id, so if a new chat is created a websocket connection is opened. If the server goes down, we need to look at all the chats within that geographical region that are in the nosql database, and restart those websocket brokers.
Sql database for user server trades consistency for latency. But we need consistent data for our user database.
No sql database for chat server will reduce latency for heavy reads. Our LRU cache based on chat_id primary key will also be useful to reduce latency, but we are trading consistency a bit here as some of our chat data will not exist in all our data centers.
SPOF, we need more load balancers, more chat servers and user servers so the application does not break due to natural disaster etc.
No implementation for media types, video, voice messages, and images. We would need to implement a media server, Store the blob data in our s3 bucket, whilst that happens we push to a message queue like amazon simple queue service, therefore we are not waiting for this to complete. Once this does complete we update the encoded url of our nosql database plus the rest of the chat-item metadata. We can also send a notification message to that user that is subscribed to this chat-id so that on the client side the are aware. But also a we update the chat, all users of that chat should be updated with the new list item as normal.
More chat servers and user servers so the application does not break due to natural disaster etc.
We would need to implement a media server, Store the blob data in our s3 bucket, whilst that happens we push to a message queue like amazon simple queue service, therefore we are not waiting for this to complete. Once this does complete we update the encoded url of our nosql database plus the rest of the chat-item metadata. We can also send a notification message to that user that is subscribed to this chat-id so that on the client side the are aware. But also a we update the chat, all users of that chat should be updated with the new list item as normal.