Design Messaging Infrastructure for global users

Question

Design a low-latency messaging system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

### Functional Requirements
1. **Low-Latency Messaging**: The system must support real-time messaging with end-to-end latency under 100ms.
2. **Global User Support**: Enable messaging for millions of users across various geographic locations.
3. **Message Delivery Guarantees**: Ensure at least once delivery semantics, with options for exactly once delivery on demand.
4. **User Authentication**: Integrate with existing Splunk user authentication systems (e.g., OAuth2).
5. **Message Types**: Support various message formats (e.g., JSON, XML).
6. **Scalability**: The system should support scaling horizontally to handle increasing loads.

### Non-Functional Requirements
1. **Availability**: Aim for 99.99% uptime with effective failover strategies.
2. **Security**: Messages must be encrypted in transit and at rest.
3. **Monitoring & Alerting**: Implement robust monitoring for system health, message delivery, and latency metrics.
4. **Performance**: Handle at least 10,000 QPS (queries per second) at peak traffic.

To estimate capacity, we consider the following:
- **User Base**: Assume 1 million global users.
- **Message Frequency**: Estimate an average of 10 messages per user per minute.
- **QPS Calculation**: This results in 1 million users * 10 messages/user/minute / 60 seconds = ~166,667 QPS.
- **Scalability Factor**: To ensure we can handle spikes, we’ll aim for a peak capacity of 200,000 QPS.
- **Storage Needs**: If we assume each message is 1 KB, peak storage requirements would be 200,000 messages * 1 KB = ~200 MB per second, translating to about 17 TB per day.

### Component Diagram
1. **API Gateway**: Use **AWS API Gateway** to handle incoming requests and route them to appropriate services.
2. **Message Queue**: Implement **Apache Kafka** for message brokering, providing high throughput and low latency.
3. **Microservices**: Deploy microservices on **Kubernetes** for message processing, ensuring scalability and fault tolerance.
4. **Database**: Utilize **Cassandra** for message storage, enabling high write throughput and availability.
5. **Cache Layer**: Implement **Redis** for caching frequently accessed messages to enhance performance.
6. **Monitoring**: Use **Prometheus** and **Grafana** for system monitoring and alerting.

### Schema Design
1. **Users Table**: `user_id (UUID, PK), username (String), authenticated (Boolean)`  
2. **Messages Table**: `message_id (UUID, PK), user_id (UUID, FK), content (Text), timestamp (Timestamp), status (Enum: sent, delivered, read)`  
3. **Topics Table**: `topic_id (UUID, PK), topic_name (String), created_at (Timestamp)`  
4. **Access Patterns**: 
   - Users can retrieve messages by `user_id` and `timestamp` for chronological retrieval.
   - Messages can be queried by `status` to track delivery states.

### Architectural Decisions
1. **Consistency vs. Availability**: Using **Cassandra** offers high availability and partition tolerance but weak consistency (eventual consistency). For messaging systems, this trade-off is acceptable since slight delays in consistency are tolerable.
2. **Message Delivery Guarantees**: Opting for at least once delivery incurs the overhead of handling duplicates but ensures no message loss. This is crucial for user messaging.
3. **Caching**: Using **Redis** as a caching layer can lead to potential stale data issues, but it significantly enhances read performance, which is more critical in a messaging app.
4. **Monitoring & Alerts**: Building in extensive monitoring increases operational complexity but is essential for maintaining the health of a global messaging service.

Design Messaging Infrastructure for global users

Splunk

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Functional Requirements

Capacity Estimation

Submit Your Answer

Splunk Product Manager Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search