System requirements

Functional:

List functional requirements for the system (Ask interviewer if stuck)...

1. User uploads videos
2. Videos storage, encoding, processing in chunks, diff resolution support, distribution to global.
3. Metadata storage with Videos' urls
4. Video streaming on client side
5. Content search/navigation

Non-Functional:

List non-functional requirements for the system...

1. Storage availability -> replica
2. Streaming stability and latency
3. Scalability of upload, stream
4. Hot videos global access
5. Network Ingress / Egress high

Capacity estimation

Estimate the scale of the system you are going to design...

Size of a video file: 750M

Total 1B videos: 750PB

DAU: 100k uploader + 1M watcher: 100QPS

Storage: Compress to 250M * 50k uploads = 1.25T/day

Latency of streaming: 400ms (for the first 1min)

API design

Define what APIs are expected from the system...

uploadVideo(userID, videoFile, metaData)

streamVideo(userID, videoID, ts)

searchVideo(userID, query)

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...

metaDataDB: unstructured metadata in doc storage like MongoDB for scalability or ACID-compliant

videoChunkDB: id-S3 location

PUSH-based CND (CloudFront) to preload hot movies

High-level design

You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design...

client: upload, read request

load balancer: forwards requests to server

authorization service: auth

Server: read and write to metadata db; retrieve S3 url from CDN or from metadata db for S3 chunks; asks Zookeeper for a write node id; stateful write and respond to client

Stateful node: store raw video in an S3 and send S3 URL to an AWS SQS.

Encoding worker: retrieves task from SQS; process video into S3 chunks; update metadata db.

CDN: stores hot videos

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

encoding and decoding: resolution choices

CDN: chunks small enough to buffer in client's memory for smooth play; stores link to S3 URL of the chunk

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

1. Write latency vs consistency and availability - SQS
2. Eventual consistency for writing

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

S3

CDN

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?