System requirements

Functional:

• Watching & uploading videos
• Search for content

Non-Functional:

• High availability
• Videos shoud play with minimal buffering
• Only one video quality available
• Security (check uploads)

Capacity estimation

• 10M daily active users
• 5 views/day/user
• 50M daily viewedvideos
• 100K daily uploads
• 200MB per video on average
• Total of ~20TB of new data each day

API design

GET /v1/videos/{id}/watch/{chunk_id}

output: { chunk data }

POST /v1/videos/upload

input: { video data }

GET /v1/videos/search

inputs: [list of keywords]

output: { list of content }

Database design

Videos table

• ID
• Name
• Description
• OwnerID
• StoreID
• UploadDateTime

High-level design

3 services so we can scale each feature independently:

• Watch service: returns videos chunks for streaming
• Upload service: queue encoding jobs
• Search service: perform full text search

Request flows

• Client A uploads a video using the upload service
• Upload service creates a new encoding job and add it to the encoding processor queue
• Encoding processor fetches job to do from the queue
• Client B calls the search service to discover content
• Search service returns a list of videos matching its search terms
• Client B calls repetitively the watch service to get the different chunks of the actual video data

Detailed component design

• The encoding processor
• Search service

Trade offs/Tech choices

• We use a 3rd party provided object store like Amazon S3 or a equivalent service to store the chunks of videos because it can scale as much as we need
• Videos metadata are indexed in a database that supports full text search like ElasticSearch

Failure scenarios/bottlenecks

• Processing video upload asynchronously avoid bottleneck if many users want to upload video at the same time
• If the external malware checker is unavailable, we cannot process upload anymore. We can use a second backup provider to in order to minimize this risk

Future improvements

• Support differents bitrates/encoding
• Recommendation system
• Liking/commenting
• Live streams
• Analytics & monitoring