System requirements

Functional:

• User is able to upload videos
• User is able to watch/stream videos in different quality
• User is able to search the videos by key words

Non-Functional:

• Availability > consistency
• Low latency for search and watch videos( search < 200ms, watch videos - less time for loading the videos < 2s)
• support large amount of users
  • 1M videos uploaded daily
  • 100M DAU
  • 1B videos watched daily
  • 100M searches daily
• support resume and upload

Capacity estimation

• Storage: 1M * 200MB * 365 = 80PB for storing the videos data
• Other data
  • user, video metadata
    • user: 8 bytes * 10 columns
    • video metadata: 8 bytes * 20 columns
  • About 80B * 300M = 24GB for user info data
  • About 160B * 1M * 365 * 5 = 320GB for video metadata

API design

• POST /uploadVideo -> 200

{

Video,

VideoMetadata

}

• GET /video/?video_id -> Video(chunk)

• POST /searchVideo/?query={query} -> VideoMetadata[]

Database design

Video data will be chunked and stored in S3.

Video Metadata will be stored in Cassendra to support high availablity and scalibility.

User data will be stored in Cassendra as well.

DB schema

Video Metadata

{

video_id, string,

name, string,

uploadBy, string,

duration, string,

manifest, Array {

chunk_id, string,

blob_uri, string,

format, string,

quality, string,

}

}

User Info

{

user_id,

user_name,

email,

gender,

...

createdAt,

updatedAt,

}

High-level design

client -> LB -> uploadService. -> user table

|

S3. <- postprocessing videoMetadata table

|

-> videoService -> /

-> searchService -> ElasticSearch.

Request flows

Upload video:

1. client enrich the request based on video data, chunk it to X chunks and generate chunk_id and manifest file.
2. client send request and hit load balancer, splitting the traffic by user_id.
3. Upload request hit uploadService.
   • Talk to S3 to get several pre signed uris for uploading
   • Uploading the chunks to pre signed uris.
4. Postprocessing service will read the uploaded chuncks and transcode to different format and qualities, store them in different blob uri and update the manifest file.

Watch video:

1. client send request to LB, LB split the traffic to different videoService.
2. VideoService query the VideoMetadata db to get the manifest and chunk uris.
3. VideoService start downloading the first chunk and return.

Serach video:

1. client send request to LB, LB split and send to searchService.
2. Service service query ElasticSearch for quick look up.
3. Return the List of VideoMetadata.

Detailed component design

Postprocessing service will be triggered by a new chunk being uploaded.

In watch service, service will based on the network and device info to auto pick up the right quality for users. If user choose to stream at a resolution, we will download the corresponding chunks for them.

In search service, the elastic search is indexed based on the videoMetadata like video title.

Trade offs/Tech choices

1. Using postprocessing to generate different format VS generate different chunks
   1. first choice is not able to fulfill the needs to stream the video
   2. second choice will be more flexible but requires a pipeline to process data
2. Postprocessing start after the whole video is uploaded VS start after the first chunk is uploaded
   • first choice will cause more latency for uploading
   • second choice may cause some unused files if an upload didn't finished.
3. Using S3 vs build own Blob storage
   • S3 is high available but introduce a external dependency
   • Building own blob storage will be additional eng effort and cause maintainence overhead, but it may be more responsive when incidents happend.

Failure scenarios/bottlenecks

If Upload/watch/serach service is down, we will have a couple of backup service.

If databased is down, we can switch to replica and restore the db.

If elasticSearch is down, we can switch to secondary ES.

If S3 is down, we may experience a problem. We can have a backup solution like building own blob storage or have another vendor as a backup.

Future improvements

We can use CDN, to cache most popular videos and decrease the latency for user to stream the videos.

We can have another databased called user watch history. This can be used for analytics for user preferences to build a recommendation system.

and can be used to support resume the video in different devices for users.