Loading...
List functional requirements for the system (Ask the chat bot for hints if stuck.)...
1) User Authentication Allow Users to create accounts,log in and manage their profiles.
2)Video Upload Enable users to upload videos in
3) Stream videos smoothly to users based on their device and network conditions
5) Comments and Likes: Allow users to comment on videos and like them.
List non-functional requirements for the system...
1) System should be highly available and fault tolorent
2) Sysetm should have low latency
3) System should be highly scalable
Estimate the scale of the system you are going to design...
Total number of users how have ever logged into youtube in a month=1000,000,000
Number of active users =1000,000 per second
let's assume if a 64 core processor server can handle 10,000 request than number of servers required =1000,000/10,000 = 100
Define what APIs are expected from the system...
1) POST /v1/video This api will be used to upload a video it will contain the video metadata and other video details
2) GET /v1/video?id= This api will be used to get the particular video.
3) Post /v1/video This api will be used to add comment in a vido it will contain the request body which will have the details of the video.
4) /v1/video?id= This api will be used to get all the comments for a video
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
For storing the metadata
For storing the metadata for particular videos or comment we can use a sql database like MySQL and for storing video will be using blob storage.For stroing metadata will have a simple videos tables which will have the details of the videos like the video description, video title and other details will have one more table that will store comments for a particular video.
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
User requests are first routed to a load balancer, which then directs them to specific web servers, and subsequently, the requests are forwarded to application servers. For trending videos within users' regions, content is stored on a CDN to be served directly from there, enhancing delivery speed.
When a user uploads a video, the video metadata is stored in a SQL database, while the actual video content is stored in blob storage. Instead of uploading the entire video at once, the video is split into fixed-length segments, allowing for easier recovery in case of failure, as only the failed segment needs re-uploading. Videos are also encoded into various formats to deliver optimal quality based on the user’s internet speed.
To scale the SQL database, sharding is used based on the first character of the video title, with each SQL server handling metadata for videos beginning with specific characters. This method facilitates database scalability.
To manage file uploads efficiently, Kafka is used, where each video chunk is assigned to a specific partition within a topic based on the video name. Consumers can then fetch and upload these chunks. For storing thumbnails, Google’s Bigtable is used, as it is well-suited for handling a large number of small files, each under 10 MB.
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
User requests are first routed to a load balancer, which then directs them to specific web servers, and subsequently, the requests are forwarded to application servers. For trending videos within users' regions, content is stored on a CDN to be served directly from there, enhancing delivery speed.
When a user uploads a video, the video metadata is stored in a SQL database, while the actual video content is stored in blob storage. Instead of uploading the entire video at once, the video is split into fixed-length segments, allowing for easier recovery in case of failure, as only the failed segment needs re-uploading. Videos are also encoded into various formats to deliver optimal quality based on the user’s internet speed.
To scale the SQL database, sharding is used based on the first character of the video title, with each SQL server handling metadata for videos beginning with specific characters. This method facilitates database scalability.
To manage file uploads efficiently, Kafka is used, where each video chunk is assigned to a specific partition within a topic based on the video name. Consumers can then fetch and upload these chunks. For storing thumbnails, Google’s Bigtable is used, as it is well-suited for handling a large number of small files, each under 10 MB.
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
User requests are first routed to a load balancer, which then directs them to specific web servers, and subsequently, the requests are forwarded to application servers. For trending videos within users' regions, content is stored on a CDN to be served directly from there, enhancing delivery speed.
When a user uploads a video, the video metadata is stored in a SQL database, while the actual video content is stored in blob storage. Instead of uploading the entire video at once, the video is split into fixed-length segments, allowing for easier recovery in case of failure, as only the failed segment needs re-uploading. Videos are also encoded into various formats to deliver optimal quality based on the user’s internet speed.
To scale the SQL database, sharding is used based on the first character of the video title, with each SQL server handling metadata for videos beginning with specific characters. This method facilitates database scalability.
To manage file uploads efficiently, Kafka is used, where each video chunk is assigned to a specific partition within a topic based on the video name. Consumers can then fetch and upload these chunks. For storing thumbnails, Google’s Bigtable is used, as it is well-suited for handling a large number of small files, each under 10 MB.
Explain any trade offs you have made and why you made certain tech choices...
1) Using sql over mysql will guarantee consistency.
Try to discuss as many failure scenarios/bottlenecks as possible.
1) Nosql databse does not gurantee consistencly so there can be some comments miss.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?
1) We can include recommendations system which will automatically movies to user.