System requirements

Functional:

• Register/login
• Send/receive real-time messages
• Offline delivery
• Message persistence
• Delivery acknowledgment

📌 Non-Functional:

• Real-time (low latency)
• Highly available
• Horizontally scalable to billions of users
• Secure (eventually support end-to-end encryption)

Capacity estimation

Assume:

1. The daily active users (DAU) is 500 million, average 10 messages have been sent, 1 / 10 is video, and 2 /10 are photos
2. Every text message is 256 Bytes, every photo is 1MB, every video is 5MB

Storage:

500m * (7 * 256B + 2 * 1MB + 5MB) = 4PB / day

Bandwidth

4PB / 86400 s = 46GB/s

send + receive: 46GB/s * 2 = 92 GB/ s

API design

/message

POST sendMessage(message_id, group_id, sender_id, receiver_ids, create_time, type, text, media_object=none, document=none)

GET getMessage(user_id)

/file

POST uploadFile(file_type, file);

GET downloadFile(user_if, file_id);

/list

GET getMessagelist(group_id, timeSpan) 分页

/group

POST group(group_id, users{}, owner_id, create_time)

GET group(group_id)

Database design

We need to store the metadata of groups, users and messages

For groups' and users' information, we can use the RDB (MySQL, PostgreSQL) because they are structured, but consider we will support 1B users; RDB is hard to horizontally scale. A NoSQL database like Cassandra would be better.

For the messages, we could also use Cassandra to store them, but for the media, we need to use a blob database.

Users:

userId

userName

createTime

lastLogin

Group:

groupId:

groupName

createTime

ownerId

members

High-level design

We have 3 ways to implement message sending and receiving:

1. pull: we will need a lot of request for the scenario, and it can not support the offline scenario
2. Long pull: better than pull, can support offline, but it still needs a lot of requests.
3. WebSocket: the best approach. When the connection is built, users can real-time connect each other during the session, quickly and efficiently

Components

ApiGateway: user auth

CDN: frequently used media

DB: for users and group metadata

Blob DB: media

cache: frequently used media and message

Load balancer: balanced forward the request to different server

Message queue: for those offline message

1. The user 1 and user 2 create a communication channel with the chat server.
2. The user 1 send a message to chat service, when the chat server receive the message, will ackowledge user 1 the message has been received.
3. The chat service store the message to a DB and forward the message to user 2 if it is online.
4. If user 2 is offline, chat server will hold the message to a transient until user 2's status is online
5. When the user 2 receives the message, it will also send an acknowledgment to the chat service.
6. The chat service acknowledges user 1's message has been successfully delivered
7. When user 2 reads the message, the application notifies the chat server.
8. The chat server notifies user 1 that user 2 has read the message.

Request flows

1. The user 1 and user 2 create a communication channel with the chat server.
2. The user 1 send a message to chat service, when the chat server receive the message, will ackowledge user 1 the message has been received.
3. The chat service forward the message to user 2 if it is online.
4. If user 2 is offline, chat server will hold and store message to a DB until user 2's status is online
5. When the user 2 receives the message, it will also send an acknowledgment to the chat service.
6. The chat service acknowledges user 1's message has been successfully delivered
7. When user 2 reads the message, the application notifies the chat server.
8. The chat server notifies user 1 that user 2 has read the message.

Detailed component design

Connection Layer

• Each client opens a persistent WebSocket to a regional connection server.
• These servers are stateless and only manage socket lifecycle + basic validation.
• They are sharded by user_id % N to scale.

📨 Message Routing Service

• When a user sends a message:
  • It hits the message router
  • If the recipient is online → forward to their connection server
  • If not → store in DB, notify via push

📚 Message Persistence

• Messages are stored in NoSQL (e.g., Cassandra), partitioned by user_id, sorted by timestamp.
• Messages are append-only for immutability and audit.
• Use Kafka or similar pub-sub to:
  • Decouple sender and receiver paths
  • Retry delivery
  • Ensure durability

⚡ Cache

• Use Redis to cache:
  • Recent messages
  • User metadata
  • Presence info
• TTL controls memory usage

🔔 Push Notifications

• Use Firebase/APNs to wake up clients when offline.

Trade offs/Tech choices

• CAP Trade-off: Prefer availability over strong consistency for messaging, using eventual consistency and ack-based delivery.
• 🧠 Caching boosts speed, but adds invalidation complexity.
• 🔐 End-to-end encryption limits server-side analytics but ensures privacy.

Failure scenarios/bottlenecks

Horizontal Scaling

• Stateless services allow replication and autoscaling
• DBs and queues are partitioned
• Object storage handles media at scale

Fault Tolerance

• Retry with exponential backoff
• Geo-redundancy for DB & Kafka
• Use quorum writes for critical metadata

Global Distribution

• Deploy in multiple regions
• Geo-route users to nearest data center
• Cross-region sync via eventual consistency

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?