System requirements

Functional:

Users can do the following activites:

• Watch a video
• Upload a video
• Search for a video

Non-Functional:

• Reliability, we don't need that a user uploads a video that gets corrupted or disappears.
• Scaling, 1 billion DAU.
  • views: If each user is watching 5 videos per day, then it'll be around 5 billion videos watched per day (5 billion read operations)
  • uploads: we can consider the upload ration to watching is 1:100, then it's 1 % of 5 billion videos, so it'll be 50 million videos uploaded per day
  • searches: i assume if it's 5 billion videos watched per day, then there is at least around 10 billion searches, since each user will search at least twice to reach the right video (this number can be higher but since there's users that go to direct links or get their video from the 1st searching attempt)
  • average video length: the average video length can be 5 minutes
• Minimize Latency, the user should stream the video with minimal delay, even with high internet bandwidth he should experience no wait time
• Favour availability over consistency, for example the subscribed channels have to always appear on a user's homepage, however it's not essential that an uploaded video, a couple of seconds ago, appears to my feed in real-time.

API design

Designing the API for a video sharing platform like YouTube or Netflix involves defining the key endpoints and their functionalities, based on the requirements you have. Below are some considerations and suggestions for your API design:

Key Functional Requirements

1. User Authentication: Users must be able to sign up, log in, and manage their accounts.
2. Video Uploading: Users need to upload videos and receive a presigned URL for direct storage access.
3. Video Metadata Management: After uploading, video details should be stored, and users should access their uploaded videos' data.
4. Video Streaming: Users should be able to watch videos smoothly.
5. Search Functionality: Users should be able to search for videos by title, description, tags, etc.
6. View Count and Analytics: After watching, users can increment view counts, and analytics data can be gathered.

Suggested API Endpoints

1. User Management

• POST /users/signup: Create a new user account.
• POST /users/login: Authenticate a user and return an access token.
• GET /users/profile: Retrieve user profile information.
• PUT /users/profile: Update user profile data.

2. Video Management

• POST /videos/upload: Initiates video upload and returns a presigned URL for uploading.
• POST /videos: Submit video metadata after upload completion (like title, description).
• GET /videos/{video_id}: Retrieve metadata for a specific video.
• GET /videos: List videos based on filters (like most viewed, latest uploads).

3. Video Streaming

• GET /videos/{video_id}/stream: Provide the manifest file for adaptive streaming.
• GET /videos/{video_id}/thumbnails: Retrieve the video's thumbnail.

4. Search Functionality

• GET /search: Search for videos by query parameters (title, tags, etc.).

5. Analytics & Interaction

• POST /videos/{video_id}/view: Increment the view count for a specific video.
• POST /videos/{video_id}/like: Like a video.
• GET /videos/{video_id}/analytics: Retrieve analytics data for a specific video (optional).

Design Considerations:

1. Authentication: Use OAuth or JWT for securing endpoints that require user authentication.
2. Rate Limiting: Implement rate limiting to prevent abuse of your API.
3. Input Validation: Ensure that all incoming data is validated and sanitized.
4. Caching: Employ caching for frequently requested resources, like video metadata or search results.
5. Error Handling: Standardize error responses across your API to improve client-side handling.

High-level design

Since we have 1 billion DAU, and we need to scale for users' uploads, views & searches, then we need to have the following components:

• Load Balancer, with multiple App servers
• Original Video Object Storage (Blob storage)
• Metadata storage (NoSQL - MongoDB)
• Message Queue for Encoding process
• Encoing service
• Encoded Video blob storage
• CDN

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...