Architect a low-latency Chat Engine

Question

Design a low-latency chat system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

- **Functional Requirements**:
  - Real-time messaging between users.
  - Support for group chats and direct messages.
  - Message history retrieval.
  - User presence indicators (online/offline).
  - Notifications for new messages.
  - Message delivery and read receipts.

- **Non-Functional Requirements**:
  - Low latency (< 100 ms for message delivery).
  - High availability (99.99% uptime).
  - Scalability to handle millions of concurrent users.
  - Data durability with eventual consistency for chat messages.
  - Security features like end-to-end encryption.

- **User Base**: Assume we target 10 million active users.
- **Messages per User**: Estimate each user sends 20 messages per day on average.
- **Total Messages**: 10,000,000 users * 20 messages = 200,000,000 messages per day.
- **Messages per Second (QPS)**: 200,000,000 messages/day / 86,400 seconds/day = ~2,314 QPS.
- **Storage Needs**: Assuming each message averages 1 KB, total storage = 200,000,000 KB/day = ~200 GB/day. Over a year, this results in ~73 TB of messages.
- **Bandwidth**: Average message payload (including metadata) is ~1 KB, resulting in ~2.3 GB of bandwidth per second.

### Component Diagram:
- **Frontend**: Web and mobile clients using React and React Native.
- **Load Balancer**: NGINX for distributing incoming traffic.
- **WebSocket Server**: Node.js with Socket.io for real-time messaging.
- **Message Broker**: Apache Kafka for message queuing and stream processing.
- **Database**: NoSQL database (Cassandra) for storing messages with wide-column storage.
- **Cache Layer**: Redis for caching recent messages and user presence data.
- **Monitoring**: Prometheus for metrics collection and Grafana for visualization.
- **Alerting**: PagerDuty for operational alerting.

### Schema Design:
- **Users Table**:
  - `user_id`: UUID (Primary Key)
  - `username`: String
  - `status`: Enum (online/offline)
  - `last_seen`: Timestamp

- **Messages Table** (Cassandra):
  - `message_id`: UUID (Primary Key)
  - `chat_id`: UUID (Partition Key)
  - `sender_id`: UUID
  - `content`: Text
  - `timestamp`: Timestamp

- **Chat Groups Table**:
  - `chat_id`: UUID (Primary Key)
  - `members`: List<UUID>

### Access Patterns:
- Efficient retrieval of messages for a specific chat using `chat_id`.
- User presence and status updates stored in Redis for fast access.

- **Consistency vs. Availability**: Opting for eventual consistency allows for low-latency message delivery but may result in temporary inconsistencies in message states. This is acceptable for chat applications where immediate consistency is not critical.

- **Caching Strategies**: Using Redis as an in-memory cache for user presence and frequently accessed messages reduces latency but introduces challenges with cache invalidation.

- **Partitioning and Sharding**: Cassandra's built-in sharding is advantageous for horizontal scalability. However, this can complicate cross-chat queries and require careful design of query patterns.

- **Monitoring and Alerting**: Implementing robust monitoring with Prometheus is essential for operational visibility. However, the overhead of monitoring can increase latency slightly, requiring a balance between detailed metrics and performance.

Architect a low-latency Chat Engine

Snowflake

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Capacity Estimation

Submit Your Answer

Snowflake Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search