Loading...
List functional requirements for the system (Ask the chat bot for hints if stuck.)...
- users can publish photos
- users sees friends published photos in reverse chronological order
- users can go to a discover page to see recommended photos
-users can like photos
-photo owner is notified when someone likes their photo
i've left out chat intentionally to reduce scope ive also left out seeing following posts in other orders to reduce scope. would this be okay for this problem?
List non-functional requirements for the system...
Estimate the scale of the system you are going to design...
User base estimation - 1 billion monthly active users. Half of them use it per day.
Posting is a lot less these days. 5% of daily users post a picture a day. 25 million posts daily. -> 300 pictures a second. On average. Peak traffic * 2.
Feed access - 500 million daily users * 10 access to feed. = 5 Billion requests for feed a day. 58k requests per second
Photo size = 2MB, we will have to store 50TB a day. Petabytes worth of data over time.
Define what APIs are expected from the system...
upload image to S3, S3 handles massive scale effectively, streaming large data.
POST /image { imageS3Uri: ..., caption: ... }
GET /upload-url for presigned URL for S3
response {status: ..., message: ... }
For S3 we will use pre-signed URL, we get S3's speed and efficiency for free, while this is also secure.
GET /feed/
GET /discover/
infer userId from headers (auth token) for these 2
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
Photo storage. We need massively scalable object stores for photos, so S3 is a good option. It also offers great upload and download speeds. Offers all that we need to deal with large files at scale.
Users Table: user_id, profile_info1, profile_infon, followers
Posts Table: post_id, user_id, caption
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
We should have multiple types of clients. Lets start with mobile and web.
for posting, we should have a posting service. behind AWS API gateway (handles authentication). it has 2 APIs, upload-url and image-post. The web services behind a load balancer will interact with S3, as well as writing to DB. As well, we should have a fanout on write for users with less than 100k followers, where we update their feed cache.
For the feed flow, the feed service should get feed information from the cache. Or for each high follower person, read their most recent posts and merge it with the cache.
For discover page, users will hit a embedding generation service that displays top matches for the user. The entity embeddings are refreshed on a cadence,
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
Explain any trade offs you have made and why you made certain tech choices...
Try to discuss as many failure scenarios/bottlenecks as possible.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?