Loading...
We need a way for users to see a specific video
We need a way for users to upload a video
We want to prioritize latency and availability of the system.
Estimate the scale of the system you are going to design...
Imagine there are 100 million active users on youtube a month, let say each of them watches about 10 videos a week or 40 videos a month. That is a total of 4 billion read requests a month. We can negate how many write requests as this probably doesn't happen as often.
But in terms of database lets say there are 2 million videos uploaded monthly, With each video being around 1gb of data, thats roughly 2 petabytes of storage on something like S3 we need monthly. And assuming 2 million requests each have some meta data about the video, we can say each request will have 1kb of meta data so roughly 2 gigabytes of database storage monthly.
For creating a new video we can have a post request to https://youtube.com/create_video. We will give userid, title, description, the actual raw video data. We will return a 302 code when the upload begins, and a 400 code if the upload fails. At this point we establish a websocket connection with the server that sends us a notification when the video finishes uploading.
For seeing a video we can have a get request https://youtube.com/:id. Here we will use the video id parameter which will normally be encoded to prevent sql injection attacks with that video id on the server. We will get a 200 code that gives us the video url, in which case then the client will request this video from a cdn. Or we return a 404 error for invalid links or videos that have not been found on our server.
we will use a no sql database because this will be a read heavy use case. We can also denormalize any data as we scale so that we make sure not to make additional queries and lower our latency. For storage we are using something like Amazon S3 to store the video files.
So we have a load balancer to distribute the load to the server. We use a LRU redis cache, and a nosql database which will hold the file url. We then have an s3 bucket to hold the raw data of the videos. We also have a file compresser that then takes the raw files and compresses them then storing them in different resolution formats 256, 512 etc stored into a our S3 and a second nosql database. We also have a background service worker that takes the files from this second database and updates our content data networks periodically.
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
Explain any trade offs you have made and why you made certain tech choices...
Try to discuss as many failure scenarios/bottlenecks as possible.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?