Design a large-scale Messaging Platform

Question

Design a multi-tenant messaging system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

### Functional Requirements:
1. **Multi-Tenant Support**: Ability to handle multiple organizations, each with their own users and data.
2. **Real-time Messaging**: Users should be able to send and receive messages instantly.
3. **Message Storage**: Persist messages for retrieval and compliance.
4. **User Presence**: Indicate online/offline status for users.
5. **Notifications**: Push notifications for new messages.

### Non-Functional Requirements:
1. **Scalability**: Should handle millions of concurrent users and requests.
2. **Low Latency**: Message delivery should occur within 100ms.
3. **High Availability**: 99.9% uptime with minimal downtime for maintenance.
4. **Data Security**: Ensure data encryption both in transit and at rest.
5. **Compliance**: Meet data protection regulations such as GDPR.

Assuming we have 10 million users, each sending an average of 10 messages per day:
- **Total Messages per Day**: 10 million users * 10 messages = 100 million messages/day.
- **Messages per Second**: 100 million messages/day / 86,400 seconds/day ≈ 1,157 messages/second.

To cater for peak traffic, we can assume a spike of 5x during peak hours:
- **Peak Messages per Second**: 1,157 messages/second * 5 ≈ 5,785 messages/second.

Given these numbers, the system should handle at least 6,000 messages per second reliably.

The architecture consists of the following components:
1. **Client Applications**: Web and mobile clients for user interaction.
2. **API Gateway**: Acts as a single entry point for all client requests, handles authentication and routing.
3. **Message Service**: A microservice responsible for sending, receiving, and storing messages (e.g., built using Node.js).
4. **Database**: A NoSQL database like MongoDB for message storage, allowing for flexible schema and horizontal scaling.
5. **Real-time Messaging**: Use WebSockets for real-time communication.
6. **Notification Service**: Push notifications managed through a service like Firebase Cloud Messaging.
7. **Load Balancer**: To distribute traffic across multiple instances of services.
8. **Monitoring and Logging**: Tools like Prometheus and Grafana for system health monitoring and logging.

### Message Schema:
```json
{
  "messageId": "UUID",
  "senderId": "String",
  "receiverId": "String",
  "content": "String",
  "timestamp": "Date",
  "status": "String"  // e.g., sent, delivered, read
}
```
### User Schema:
```json
{
  "userId": "String",
  "username": "String",
  "status": "String",  // e.g., online, offline
  "lastActive": "Date"
}
```
### Access Patterns:
- Frequent reads for retrieving messages.
- Updates for message status.
- Writes when sending messages.

1. **Consistency vs. Availability**: Opting for eventual consistency to ensure high availability. Messages may not be immediately visible to recipients, but the system remains responsive.
2. **Latency vs. Durability**: Storing messages in memory (e.g., Redis) for quick access but also persisting them to a database for durability. This can introduce some latency when persisting data.
3. **Complexity vs. Scalability**: Using microservices increases complexity but allows for better scalability as different services can be scaled independently based on load.
4. **Cost vs. Performance**: Choosing a NoSQL database to optimize for write-heavy operations while accepting potential trade-offs in complex querying capabilities.

Design a large-scale Messaging Platform

OpenAI

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Functional Requirements:

Capacity Estimation

Submit Your Answer

OpenAI Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search