Loading...
Capacity Estimation:
About 500M users daily
About 20% upload a reel, and maybe they upload 2 photos and 1 video. So about 100M uploading, for 5MB for each photo, 30MB for video for 30 secs.. so about 40MB per user. 100M * 40MB = 4000M MB ~ 4M GB ~ 4000TB ~ 4PB.
So about 4PB of storage is needed.
Fetch Content: About let's say, we fetch 50 reels at 1 time, very min we call fetch to fetch more. About 1.5 Hr of usage ~ 100 fetch per user per day * 500M = 50000M fetches a day ~ 0.5M TPS
So about 500K tps is needed for recommendation system.
Latency Targets: 1 sec for fetching latest content i.e. at most it should take about 1 sec to fetch the reels metadata to the app, the app can then down the rest of the media. Each call should suggest ~50 reels. So this acts like a Batch API which suggest 50 things and then when user is getting close towards the end of the cached reels on app, App will call this API and start downloading more content.
Eventual consistency is OK for fetched content.
Some rough components that would be needed:
1) Blob Storage for storing reel.
2) Metadata DDB.
3) Comments and Likes DDB.
4) Recommendation System for fetching reels.
5) User db for storing user info and list of people/accounts you follow.
6) Internet Scale is needed as users will grow substantially, so large PB storage system is needed for media store and high availability and low latency system is needed for recommending reels.
Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...
User APIs:
POST /api/v1/user/create <-- Create User with some credentials.
POST /api/v1/user/delete <-- Delete user. (Authenticated)
PUT /api/v1/user/update <--Update Settings. (Authenticated)
GET /api/v1/user/read <-- Read user settings. (Authenticated).
Read User Profile:
GET /api/v1/users/profile/{id} <-- Read User Profile.
Media/POST API.
/api/v1/media/upload <-- Upload Media API.
/api/v1/media/update <--Update existing media metadata.
/api/vi/media/delete <-- Remove media.
/api/v1/media/read <-- Read media some metadata including blob presigned URL, basic count likes, # of comments etc.
Interaction APIs:
/api/v1/media/comment/post
/api/v1/media/comment/delete
/api/v1/media/comment/list <-- List/read comment thread.
/api/v1/media/like
/api/v1/media/unlike
/api/v1/user/follow
/api/v1/user/unfollow
Blob URLs:
media.instagram.com/m/
Main Feed API:
/api/v1/feed/fetch <-- Main API to request feed for the user.
Essentially CRUD APIs are provided for User Mgmt, Media Mgmt and interaction APIs are provided to follow, unfollow another user, post comments and like etc.
Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.
There are few key components:
1) App Client interacts with the Load Balancers which forward the request to the API Gateway.
2) API Gateway is responsible for rate limiting as well as authentication for the secure APIS.
3) The backend consists of 3 major sub systems:
1) User Mgmt service -> This service is responsible for providing the crud apis for user mgmt and is backed by the No SQL DB which store the user/profile information. The service also has a cache which allows to quickly retrieve users which a particular user follow, which will allows the Feed Service to quickly figure out which users post are needed to create a feed for the user. This cache is simply based on LRU principal and an entry is evicted/updated when user updates their settings like they follow/unfollow someone. In such case entry is invalidated and the system would need to recreate the entry.
The user db consists of primary key of userId and additional attributes such as other users this user follow, media/post this user has uploaded (mediaId)
2) Media Mgmt service -> This service is responsible for managing the media/images/videos that are uploaded by the user. The actual assets are stored in blob storage and managed by the metadata index db, which store information about the media like mediaId, who uploaded it, extra metadata etc. The db also has secondary index which allows it to query post/media uploaded by the user. So a secondary index (by user) is also present. This will be important for feed service and looking at the profile info.
3) The interaction service is the third component which is responsible for storing the comment etc, and when a comment/likes are stores, it update the media metadata and user db appropirately.
4) The profile service uses various api from user mgmt, media service etc to create a profile record. A cache could be added so that profile for famous people are not recomputed each time.
5) The main component is the feedservice, which is responsible for creating the feed for the user. When a user requests the feed, it first gets the users the requesting the user follows. The UserMgmtService has a Follow cache which allow it to return the users quite quickly. From there it calls the media mgmt service (which has a cache as well, for popular users), and it can return top N post for each user. The feed service then create the timeline using user meta data, timestamp etc and popularity to create the feed. The feed response includes the posts/media from the users and some additional metadata for display purposes. This component is active pull.
6) The other component is notification service. When a media is uploaded (i.e. a post is created) and published by the user, the media service can emit a msg to the notification service. The notification service on backend can query the users who follow the user who uploaded and send out the notifications to them offline through android/ios push notification system.
Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.
The main component is the feedservice, which is responsible for creating the feed for the user. When a user requests the feed, it first gets the users the requesting the user follows. The UserMgmtService has a Follow cache which allow it to return the users quite quickly. From there it calls the media mgmt service (which has a cache as well, for popular users), and it can return top N post for each user. The feed service then create the timeline using user meta data, timestamp etc and popularity to create the feed. The feed response includes the posts/media from the users and some additional metadata for display purposes. This component is active pull.
The other key thing is the media storage. The media storage uses the petabyte scale solution to store the assets. The assets index/metadata is store in db and media service allows you to retrieve the media metadata by media id or list all the media id by users which then can be used to retrieve the media meta data. The media service can also cache the result for popular users and when the media is updated the cache is also updated. This allows for fast operation including for the feed service.
The additional thing that can be added is the event bus. Each service when an operation is performed can emit an event on the event bus. For e.g. if a media is added by the user, the media service emits and event. The profile service can read this event and update the cached profile for the user. This allows for async operation including sending notifications to users who are not online in case one of their followers actually uploads a post.