Design a large-scale Video Streaming Platform

Question

Design a real-time video streaming system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

### Functional Requirements:
1. **User Authentication**: Users must be able to register, log in, and manage their accounts.
2. **Video Uploading**: Users can upload videos in various formats and resolutions.
3. **Real-time Streaming**: The system must support live streaming with minimal latency (ideally under 2 seconds).
4. **Video Playback**: Users can view videos on demand with adaptive bitrate streaming.
5. **Comments and Reactions**: Users can comment on and react to videos in real-time.
6. **Search and Discovery**: Users can search for videos by tags, titles, and categories.

### Non-Functional Requirements:
1. **Scalability**: The system should handle millions of concurrent users and scale horizontally.
2. **Availability**: Aiming for 99.99% uptime to ensure user satisfaction.
3. **Performance**: Latency for video playback should be under 2 seconds for the majority of users.
4. **Security**: Ensure secure video uploads and user data protection with encryption.
5. **Fault Tolerance**: The system should gracefully handle failures and ensure data integrity.

Assuming Figma's video streaming platform is expected to support 10 million users:
- **Concurrent Users**: Let’s estimate that 10% of users are active at peak times, giving us 1 million concurrent users.
- **Video Bitrate**: Average video quality at 2 Mbps (megabits per second).
- **Data Transfer**: For 1 million users, total bandwidth needed = 1,000,000 users * 2 Mbps = 2,000,000 Mbps = 2 Tbps (terabits per second).
- **Storage Needs**: If each user uploads an average of 1 hour of video, and videos are stored in 5 different resolutions, we can estimate:
  - Average video size per hour: 1 GB (for highest quality), 1 hour * 10 million users = 10 million GB.
Thus, we will need approximately 10 PB (petabytes) of storage for video content, considering redundancy and backups.

### Component Diagram:
- **Client Application**: Web and mobile apps for user interaction.
- **Load Balancer**: Distributes incoming user requests across multiple servers.
- **Video Ingestion Service**: Handles video uploads and transcoding.
- **Media Server**: Real-time streaming using protocols such as WebRTC and HLS.
- **Database**: Use PostgreSQL for user data and metadata, with a separate NoSQL database (like DynamoDB) for video metadata and comments.
- **Caching Layer**: Redis for fast access to frequently requested video metadata.
- **Content Delivery Network (CDN)**: For efficient video delivery and reduced latency.
- **Monitoring and Logging**: Use Prometheus for monitoring and ELK stack for logging.

### Technology Choices:
- **Frontend**: React.js for a responsive user interface.
- **Backend**: Node.js for real-time communication and handling requests.
- **Transcoding**: FFmpeg for video processing.

### Schema Design:
1. **Users Table**:
   - `user_id` (Primary Key, UUID)
   - `username` (String)
   - `email` (String)
   - `password_hash` (String)
   - `created_at` (Timestamp)

2. **Videos Table**:
   - `video_id` (Primary Key, UUID)
   - `user_id` (Foreign Key)
   - `title` (String)
   - `description` (Text)
   - `upload_date` (Timestamp)
   - `views` (Integer)
   - `status` (Enum: 'processing', 'live', 'archived')

3. **Comments Table**:
   - `comment_id` (Primary Key, UUID)
   - `video_id` (Foreign Key)
   - `user_id` (Foreign Key)
   - `comment_text` (Text)
   - `created_at` (Timestamp)

### Access Patterns:
- Frequent reads for video playback and metadata, hence the use of caching for performance.

### Architectural Decisions:
1. **Consistency vs. Availability**: Opted for eventual consistency for comments and reactions to enhance availability during high traffic.
2. **Database Selection**: Chose PostgreSQL for user data requiring ACID transactions, while NoSQL fits well for unstructured video metadata, balancing consistency and speed.
3. **Caching Strategy**: Implemented Redis for caching frequently accessed metadata, trading off some consistency for improved read performance.
4. **Load Balancer**: Using a load balancer introduces slight latency but significantly increases fault tolerance and system resilience.
5. **CDN**: Opting for a CDN reduces latency for video delivery but increases complexity in cache invalidation for new uploads.

Design a large-scale Video Streaming Platform

Figma

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Functional Requirements:

Capacity Estimation

Submit Your Answer

Figma Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search