Design a Chat for Bloomberg

Question

Design a real-time chat system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

### Functional Requirements
- **Real-time Messaging**: Users should be able to send and receive messages instantly.
- **Group Chats**: Support for both one-on-one and group chats, with the ability to add/remove participants.
- **Message Persistence**: Messages should be stored and retrievable even after users go offline.
- **File Sharing**: Allow users to share files, images, and links within chats.
- **Notifications**: Users should receive notifications for new messages, even when the app is in the background.

### Non-Functional Requirements
- **Scalability**: The system must handle millions of concurrent connections seamlessly.
- **Latency**: Message delivery should occur in less than 200ms under normal conditions.
- **Reliability**: The system should have 99.99% uptime with robust handling of failures.
- **Security**: Ensure end-to-end encryption of messages and secure user authentication.

Assuming Bloomberg has approximately 300,000 active users for the chat feature:
- **Peak Concurrent Users**: Let's estimate 10% are active simultaneously, leading to 30,000 concurrent users.
- **Messages Per User**: If each user sends an average of 10 messages per minute, the total messages would be:
  
  $$ 30,000 \text{ users} \times 10 \text{ messages/user/min} = 300,000 \text{ messages/min} $$
  
- **Storage Requirement**: Assuming an average message size of 1 KB, the storage required for 1 hour would be:
  
  $$ 300,000 \text{ messages/min} \times 60 \text{ min} \times 1 \text{ KB} = 18 \text{ GB/hour} $$
  
Thus, for a day, we would need approximately 432 GB for message storage, not including user files.

### Component Diagram
- **Frontend**: Web and mobile SDKs for users to send and receive messages.
- **API Gateway**: Handles all incoming requests and routes them to relevant services.
- **Message Service**: Responsible for sending and receiving messages in real-time (using WebSockets).
- **Database**: A distributed database like **Cassandra** for storing messages and user data.
- **File Storage**: Use **AWS S3** for storing shared files and images.
- **Notification Service**: Push notifications through services like **Firebase Cloud Messaging**.
- **ML Service**: Responsible for real-time message ranking and spam detection using models hosted on **TensorFlow Serving**.

### Schema Design
- **User Table**: Stores user information (user_id, username, password_hash, last_active).
- **Message Table**: (message_id, sender_id, receiver_id, group_id, content, timestamp, status).
  - **Indexes**: Composite index on (group_id, timestamp) for efficient retrieval of messages in group chats.
- **Group Table**: (group_id, group_name, member_ids).
- **File Table**: (file_id, uploader_id, file_url, timestamp).

This structure supports fast access patterns for both direct messages and group chats, allowing quick retrieval of recent conversations.

### Architectural Decisions
- **Consistency vs Availability**: Opting for eventual consistency in message delivery to improve availability and performance, especially under high load.
- **Real-time vs Batch Processing**: Using real-time processing for immediate message delivery but incorporating batch processing for analytics (user engagement, message trends).
- **Model Freshness**: Implementing a canary deployment strategy for ML models to minimize impact from new models while ensuring model freshness and relevance.
- **Cold Start Problem**: Addressing cold starts for new users by using collaborative filtering based on similar users' behaviors until enough data is gathered for personalized recommendations.

Design a Chat for Bloomberg

Bloomberg

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Functional Requirements

Capacity Estimation

Submit Your Answer

Bloomberg Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search