System requirements

Functional:

1. Maximum Delay of 1 Minute:
2. Different Resolutions Based on Network Bandwidth and Screen Type:
3. Engagement Tools like Live Polling and Messaging:
4. Checking Users Preferences and Age for Content Moderation:

Non-Functional:

1. Highly scalable
2. Available
3. maximum quality

under load

Capacity estimation

1. How much video will need to be processed per live stream?

Let us assume 8k footage being captured in the event. We can assume that the highest resolution devices consuming it will be 4K devices (2160p). But we can restrict ourselves to serve upto HD for live streams. Let us assume that the resolutions we want to serve are 1080p, 720p, 480p and 360p. Let us also assume that the encoding of these videos is h.264. A cricket match lasts for approximately 10 hours. A 2 hour HD movie is about 2 GB in size. We assume that a cricket match will be about 10 GB in size. => 720p footage will be about 10 / 2 = 5 GB. => 480p footage will be about 10 / 4 = 2.5 GB. => 360p footage will be about 10 / 8 = 1.25 GB. That's roughly 18.75 GB considering all resolutions. The raw video will be broken into segments, and each segment will be processed into multiple resolutions and codecs. Let us assume we have 4 codecs. That's roughly 18.75 * 4 = 75 GB.

2. How much data will be transferred from a CDN to users in a single live stream?

Assume the number of users watching a match = 10 million. Assume 50% having HD devices, and 50% having mobile 480p devices. Assume that a user watches 50% of a live stream on average. For a 10 hour match, that's 10 GB in HD. It's 10 / 4 = 2.5 GB in 480p. Total data transfer = SUM(size_of_video_type * number_of_users_for_type) * average_watch_percentage = (10 GB * 50/100 * 10^6 + 2.5 GB * 50/100 * 10^6) * 50/100 = 3.125 * 10^6 GB = 3.125 PB

API design

• Endpoint: POST /api/videos/upload
• Purpose: Allows the live stream source to upload video data to the ingest server.
• Request Parameters:
  • stream_key: Unique identifier for the stream.
  • video_data: Raw video data sent over RTMP.
  • metadata: Additional metadata like title, description, and tags.
• Response:
  • 200 OK with a success message.
  • 400 Bad Request for validation errors.

• Endpoint: POST /api/videos/transform
• Purpose: Transcodes raw video into multiple resolutions and codecs.
• Request Parameters:
  • video_id: Identifier for the video to transform.
  • target_formats: Array of desired formats (e.g., [1080p, 720p, 480p]).
• Response:
  • 200 OK with transformation status details.
  • 404 Not Found if the video is not available.

• Endpoint: GET /api/videos/stream
• Purpose: Streams the processed video to the CDN for distribution.
• Request Parameters:
  • video_id: Identifier of the video requested.
  • resolution: Desired resolution based on user’s device (optional).
• Response:
  • 200 OK with the video stream URL.
  • 503 Service Unavailable if the stream is temporarily inaccessible.

• Endpoint: POST /api/users/preferences
• Purpose: Manages user preferences and performs content moderation checks.
• Request Parameters:
  • user_id: User identifier.
  • preferences: JSON object containing user preferences like age, content restrictions.
• Response:
  • 200 OK with updated preferences.
  • 400 Bad Request for validation errors.

• Endpoint: POST /api/interactions/message
• Purpose: Facilitates real-time engagement through messaging or polling.
• Request Parameters:
  • user_id: ID of the engaging user.
  • interaction_type: Type of interaction (e.g., message, poll).
  • content: Content of the interaction.
• Response:
  • 200 OK confirming receipt.
  • 401 Unauthorized for unauthenticated requests.

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...

High-level design

• Live Stream Source: Origin of video content, utilizing protocols such as RTMP or HTTP for uploading.
• Media Ingest Server: Captures incoming streams, preparing for transcoding and distribution.
• Media Processing Service: Transcodes video into different formats and resolutions utilizing adaptive bitrate coding necessary for maintaining quality depending on bandwidth.
• Content Delivery Network (CDN): Distributes video streams globally, minimizing latency and buffering by being geographically closer to end-users.
• Viewer's Device: Receives streams, supports engagement through real-time updates using the WebSockets server.
• Engagement Features (Poll & Message Interfaces): Supports real-time interactivity, enhancing viewer's experience with features like polling and messaging.
• User Preference & Moderation: Manages user profiles, preferences, and content moderation, leveraging the content moderation engine with load balancing for efficient task management.
• Support Services:
  • User Database: Stores user profiles and preferences.
  • Analytics & AI/ML: Utilizes data for trends, providing content moderation and user personalization.
• External Services:
  • Payment Gateway: Manages subscriptions and payments.
  • Authentication Service: Handles secure user logins and permissions.

Request flows

From Live stream source via RTMP, it is received or ingested by Media ingest server performing initial processing, possibly buffering/transcoding minor segments before forwarding it. Media Ingest Server also uploads video to Distributed File System for future use.

Stream is picked up by Video Transformation service to convert into multiple resolutions or formats as required using advanced codecs.

This is then pushed to message queues where it is processed at its own pace and distributes the processed video to different CDN networks based on user location.

Detailed component design

Video Transformation service will use ffmpeg to transform incoming videos to different resolutions and codec.

Strategies to Achieve Maximum Delay of 1 Minute

1. Protocol Choice:
   • RTMP and HTTP for Ingestion: Using efficient protocols like RTMP (Real-Time Messaging Protocol) for fast video ingestion is crucial. Though RTMP is commonly used for low-latency streaming, moving towards HTTP (with protocols like HLS or DASH) can help with scalability and reliability purposes while achieving near-real-time delivery.
2. Adaptive Bitrate Streaming:
   • Use of HLS/DASH with Low Latency: Employ adaptive bitrate protocols that allow the client device to select the best possible stream quality based on its current network conditions. Low latency configurations in HLS or DASH enable segment-based streaming while maintaining lower delay.
3. Edge Caching:
   • CDN Utilization: Leveraging a Content Delivery Network (CDN) allows for caching of video segments at the edge locations that are geographically closer to the end-users. This minimizes time lost in requesting video segments from the origin server, significantly reducing delay
4. Transcoding Speed and Efficiency:
   • Parallel Transcoding Workers: Implement a parallel processing architecture with worker nodes, enabling multiple resolutions and formats to be processed simultaneously. Quick transformation helps the system keep pace with the live stream.
5. WebSockets for Real-time Communication:
   • Bi-directional, Low-latency Interaction: For engaging features like chat and polling, the use of WebSockets supports instant communication, facilitating responses within milliseconds.

Buffer Management

Client-side Buffer Management

• Minimal Buffer Duration: Set up a small initial buffer, typically around 2-4 seconds, which can be dynamically adjusted based on real-time network conditions. This offers a balance between consistent playback and low latency.
• Buffer Adaptive Algorithms: Implement algorithms to adaptively manage buffer size based on detected network performance. For instance, if the network is stable, buffer size can be slightly increased; if unstable, reduce to minimize delay.

Server-side Buffer Management

• Pre-packaged Segments: On the server side, manage buffer by preparing short-duration video segments ready for immediate delivery. This ensures rapid availability without waiting for larger chunks to fill the buffer.
• Buffer Overhead Reduction: Maintain minimal server buffer overhead by efficiently managing incoming stream data, thus keeping the processing pipeline flowing smoothly without backups.

Edge Processing

Content Delivery Network (CDN) with Edge Caching

• Local Edge Nodes: Deploy edge nodes through CDN strategies that cache and deliver content from locations geographically closer to users, reducing the travel time for each request. This ensures that requests are fulfilled promptly, significantly cutting down latency.
• Real-time Content Refresh: Ensure the CDN system is configured for frequent content refresh, so cached segments are updated in near real-time without incurring significant delays for live stream updates.

Trade offs/Tech choices

The original video will be broken into chunks of equal size, and assigned to different worker nodes in the system. As these chunks are processed into different resolutions and codecs, we push the results to a distributed file system like S3. We are mainly interested in the hot chunks of the video (last 60 seconds) as most of our viewers want to view the video live.

In this we decided to offload last mile delivery to existing CDN solutions. If we choose to build our own CDN solution, the advantages are: a. Customised as per requirements b. May be cheaper in the long run c. Higher control over it's execution, future extensions and internals The disadvantages are: a. High initial cost b. May be expensive as compared to available solutions c. Is not as well tested as existing solutions d. Existing solutions may improve and get better by the time our custom solution is launched

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?