Design Facebook Messenger

Functional Requirements:

1. Message a contact
2. Receive messages
3. Receive notifications of new messages
4. Unsend messages
5. Read receipts

Non-Functional Requirements:

1. Low Latency - users should be able to quickly receive messages after they are sent
2. High Availability (99.99%) - users should not lose the ability to communicate with their contacts
3. Scalability - the application should be able to handle high volume of messages
4. Reliability - messages should not be lost

Traffic estimation

• 100M daily active users
• Each user sends on average 50 messages a day
• The writes would probably be proportional to the reads as there is an exchange during conversations
• Each message is approx. 100 characters ie. 100B
• Total write/read requests per day: 100M * 50 = 5 Billion
• Total write/read requests per second = 5B / (24 * 60 * 60) = 5B / (30 * 3600) = 5B / (30 * 4000) = 5B / (100000) = 50K
• Total written data per day: 100M * 50 * 100B = 500GB

API design

POST /messages

{senderId, receiverId, content}

This will be for creating a new message.

PUT messages/:messageId

{userId}

This will solely be used to update the read receipt of the given message.

GET /messages?userId&contactId&page&limit

This will return all the messages that the user has yet received. We will have a page and limit parameter so that it will retrieve only a certain amount of messages at a time.

Database design

User

userId

password

email

firstName

lastName

Message

messageId

senderId

receiverId

message

timestamp

read

High-level architecture

1. We will have a real-time messaging service which will be responsible for handling when a user is online, it will push new messages to the user and also write new messages to the database.
2. We will have a notification service which will be responsible for providing new updates when a user first logs in to the app or reopens the app after being idle for a while.
3. We will have a cache which the messaging service will write new messages to if the receiving user is not online. This can be a write-back cache as the database can be eventually consistent and we can avoid write latency. If a user logs in, the client will query the notification API and the notification API will check the cache for new messages, the message can be removed from the cache once it has been read. Otherwise, we will follow an LRU eviction policy to handle any messages that are never read.
4. We will use a NOSQL database for this as we do not have complex joins or queries to worry about. We can easily scale using NOSQL.

Request flows

1. For the real-time messaging API, when a user is online, we will open a web-socket connection with the user in order to achieve low latency messaging. For messages sent to the API, if the API has an open connection with that user, it will send the message directly to the user through the web socket, then send the message to the database. If the user opens the message, the user will send a read confirmation to the API. Otherwise if the user is offline, the message will be sent to the cache and then to the database asyncronously.
2. For the notification API, when a user first logs in after being offline, the user will send a request to the notification API for new messages, the API will check the cache for any new messages for the user, if there are any, it will send them in the response. If the user is logging into a new device, but is on the same account, we will have to pull all messages for that user as the messages will not be cached on their device.

Component design

We can create a composite index on the senderId and receiverId in order to quickly find messages between two contacts.

Trade offs/Tech choices

1. We choose a NOSQL Database which will require us to denormalize the data, causing duplicate data. However, the main duplicate information will be the user ids which should be okay.
2. We have a cache which could impact the consistency between cache and the database.

Failure scenarios

1. Database goes down

Future improvements

1. We can add replicas to the database to ensure high availability.