Requirements

Functional Requirements:

• Users can upload videos.
• Users can stream uploaded videos in multiple qualities.
• Allow users to like/dislike videos.
• Support tracking video view counts
• Allow user to serach for videos

Non-Functional Requirements:

• Low latency for streaming videos (<100ms)
• Highly Available
• Support massive scale
• High read throughput ~2.5 trillion/day reads
• The system should be durable - uploaded videos are never lost - configured with resumable uploads via multipart or chunked uploads
• Cost-efficient storage

Capacity Estimation

• 1B users
• ~100M DAU
• Uploads:
  • 10% of DAU posts videos daily so:
    • Per day: ~1M uploads
    • Per sec: ~12 uploads
    • peak traffic: 5x so ~60 uploads/sec
• Reads:
  • All DAU watches ~5 videos daily and one video view = many read requests with ABR streaming
    • On Avg, 1 video = 500 segments
    • Each segment = 1 read request
    • So per day ~5B x 500 = 2.5 trillion/day reads
    • Per sec: 29 million req
• Storage:
  • Raw videos:
    • Avg video size = 500MB
    • per day: 1M uploads = 500TB/day
    • Per year: ~180 PB
    • Each video is stored in multiple
  • Transcoded videos:
    • Each video is stored in 5 qualities
    • Avg size for each quality is 0.5x raw ~250MB
    • So for 5 -> 1.75GB
    • Daily is 1M x 1.75GB = 1.75 PB/day
    • ~640 PB/year
• Total Storage per year: 180 + 640 = ~820pb
• CDN Cache Footprint - only hot videos are cached
  • ~10% videos are hot
  • CDN cache retention: ~3 days
  • Each video (all renditions + segments): ~1.5 GB average cached footprint
  • So:
    • 1M daily uploads -> 100K hot vidoes
    • Each video = 1.75gb
    • So 100k x 1.75 = 450TB cache footprint

API Design

-> User APIs

POST /users

Get /users/{userid}

-> Video upload

POST /videos/upload

• Returns pre signed url for uploading

POST /videos/upload/complete

• Marks video upload complete and pushes event to kafka

-> Video streaming

GET /video/{videoid}

• Returns video manifest url
• Manifest contains
  • available resolution
  • ABR options
  • Urls for segments (CDN)
  • segments sequence order

-> Video reaction

POST /videos/{videoid}/like

POST /videos/{videoid}/dislike

-> Search

GET /serach?q={query}

• Return video ids + metadata

High-Level Design

Acts as the entry point for all client requests.

Responsibilities:

• Authentication (if required)
• Rate limiting - Applied using token bucket per IP to prevent fraudulent or bot views
• Request routing

Load Balancer

Distributes traffic across multiple instances of all services to improve scalability and availability.

CDN

• Caching layer, storing public facing data, eg:
  • Video segments
  • Manifest files
  • Thumbnails
  • etc public facing data
• All request for this info goes to CDN first, as mentioned not all videos info is stored in CDN, only hot vidoes are stored in CDN.
• If CDN cache miss -> fetch from object storage

Object Storage

• Stores raw videos
• Stores transcoded video segments in multiple resolutions
• Stores manifest files, thumbnails etc

Upload Service

• Recieves the upload request
• Does basic validation of request
• Creates a upload session and returns a pre signed URL for client ot upload video to object storage

Async Upload Workflow

• Implemented via a workflow orchestration
  • Why? -> because the it is durable, implements retries and in case if upload fails midway, it can retry from the failed steps.
  • Flow:
    • Upload complete API triggers teh workflow
    • Workflow define teh upload step whcih executes step by step
      1. Transcode first each video in 5 different resolutions
      2. For each resolution Segment video into 10 sec chunks and save it to object storage
      3. Save each seg to object storage
      4. Extract title, description etc and update the inverted indexes.

Video Service

• Handles get video requests -> requests manifest url from redis or if cahce miss then from DB
• Also handles saving Video reactions + updating the total likes/views

Search Service

• receieves a serach query
• Parses the query and normalizes it
• Queries inverted index -> gets back video ids from it
• Merges and Passes the video ids to rank service -> gets back ranked ids
• For those ids fetches the metadata and returns response

Rank Service

• Ranks and returns teh video ids it recieves
• The ranking is based on analytics like:
  • Video interactions
  • Views
  • Likes
  • etc..

User Service

• Creates/updates user profile info

Redis

• Cahcing layer not source of truth
• used to store video metadata for quick and fast response for apis like Get Video, Search queries
• We do lazy laoding on redis cache -> that is when a very first request comes in from a video, fetch metadata from DB then store in redis so subsequent rquest will be fetched from redis
• TTL is short lived ~1hr

PostreSQL + Casandra

• Primary persistent storage
• PostrSQL -> user, furutre entities like channels, ChannleSubscribers etc
• NoSQL -> Video, VideoMetadata, VideoReaction, VideoUpload etc

Database Design

We use a hybrid SQL (PostgreSQL) + NoSQL (Cassandra) architecture.

• SQL is used for strongly consistent relational data like Users, Subscriptions, and Channels where relationships and transactions matter.
• NoSQL (Cassandra) is used for high-scale data like Videos, Metadata, and Reactions due to its horizontal scalability and high write throughput.

Data Models

User (SQL)

Video (Cassandra)

• video_id (partition key)
• user_id
• storage_url

VideoMetadata (Cassandra)

• video_id (partition key)
• title, description, tags
• like_count, view_count

VideoReaction (Cassandra)

• video_id (partition key)
• user_id (clustering key)
• reaction_type (LIKE / DISLIKE)
• created_at

ensures one reaction per user per video

Storage

• Raw videos are stored in object storage
• Transcoded video segments are stored in object storage and referenced via metadata

Search System

We use a distributed inverted index, where:

• keyword → list of video_ids
• used to support fast search queries

Partitioning Strategy

• Most NoSQL tables are partitioned by video_id to support fast video-centric reads.
• This ensures even distribution and aligns with primary access patterns (video playback, metadata fetch, reactions).

Detailed Component Design

We will now deep dive into the following topics:

Adaptive Bitrate Streaming

ABR is needed in system like this, because we dont want to return the entire video content when user requests to watch the video, that is not efficent and takes a very long time when videos are very large. Instead we implement ABR using HLS/DASH:

How it works

• User request to watch the video
• GET /videos/videoId triggers
• The API returns back a manifest url
• Client fetches the manifest file from the url. The file contains:
  • Video segments URL (CDN)
  • Segment sequences
  • Available resolution
  • Other ABR options like, segment lengths, 10sec, 20sec etc
• Client uses the video segment url and fetches the content of the segment from CDN that user is currently watching as well the current quality user has selected, for eg: if user is at 0 sec, it fetches the first segment and starts playing
• As user continues to watch, the client continuosly polls CDN to retrieve the next segment, so if user is at 1:10 min, it will poll and fetch the video segments containing 1:20 to 1:30 content.
• Even if user peeks ahead of video, the cleint will poll CDN to fetch the video segement of taht partitucal minute and play the video.

CDN

-> CDN acts as the primary delivery layer for video segments and manifests.

Flow:

Client → CDN Edge → (cache hit) → serve immediately

→ (cache miss) → fetch from object storage

-> Hot Segment Serving (Edge Optimization)

• Hot segements are frequently reqeusted segments eg. viral videos
• trending content etc
• CDN significantly improves performance by serving hot segments directly from edge cache:
  • How it works:

First user request:

• • • CDN cache miss
    • Segment fetched from object storage (origin)
    • Stored in edge cache

Subsequent requests:

• • • Same segment requested by other users
    • Served directly from nearby CDN edge
    • No origin fetch required

Upload pipeline

-> Upload initiation

• client calls POST /video/upload
• backend authenticates and creates an upload session
• Returns a pre signed URL for object storage
• Client uses the uRl and uploads the vidoe to object storage

-> Upload completion

• client calls POST /video/complete
• Backend validates that the video exists in object storage
• Marks the video state as UPLOADED
• Then it triggers the workflow

-> Worflow steps:

• Transcoding -> Transcode each video in 5 different resolutions
• Segmenting -> For each resolution Segment video into 10 sec chunks and save it to object storage
• Saving -> Save each seg to object storage
• Manifest -> Manifest file and URL is generated and saved
• Updated Index -> Extract title, description etc and update the inverted indexes.

-> Why workflow engine?

• Durability -> Saga state is persisted
• Retries -> Failed steps are retried automatically
• Checkpointing -> Last successful step is saved and failure are resumed from there
• Observability -> full trace per video

Failure handling

• If a step fails
  • Only that step is retried
  • Prev successful steps are not executed again
• If workes crashes
  • workflow resumes from last checkpoint.

Additionally, further failure handling logic could be added as custom logic, where if sacing segments failed midway, we check last saved segment from object stoarge and resuime from next one.

Counters and Analytics

We handle likes, dislikes, and views using an event-driven system instead of direct DB updates.

-> Flow

• User action → event generated (like/view/dislike)
• Events processed asynchronously

->Counter handling

• Real-time layer (Redis)
  • Keeps fast counters per video
  • Used for UI display
  • Updated via streaming/batched events
• Persistent layer (Cassandra/NoSQL)
  • Stores aggregated counters
  • Updated in batches from stream processors

->Views handling

• Views are high volume
• We aggregate them in batches (not per request write)

We avoid direct DB writes per event and use streaming + batching to scale write-heavy counters.