System requirements

Functional:

List functional requirements for the system (Ask the chat bot for hints if stuck.)...

- users can publish photos

- users sees friends published photos in reverse chronological order

- users can go to a discover page to see recommended photos

-users can like photos

- The system should push or promptly show notifications for events

i've left out chat intentionally to reduce scope ive also left out seeing following posts in other orders to reduce scope. would this be okay for this problem?

Non-Functional:

List non-functional requirements for the system...

• Followers should see posted photos in the order or seconds. Eventual consistency probably... there may be a slight delay for users to see the post, which is okay.
• Users should interact with APIs with authenticated token, so that their account is secure and we get the userId so that we can display their follower information and recommendations
• Service should be pretty available, people should be able to use it 99% of the time
• Data should be durable, users should not see that their posted pictures are gone
• feed loading should be fast, we shouldn't see spinning too much
• service needs to handle millions of active users and billions of accounts.

Capacity estimation

Estimate the scale of the system you are going to design...

User base estimation - 1 billion monthly active users. Half of them use it per day.

Posting is a lot less these days. 5% of daily users post a picture a day. 25 million posts daily. -> 300 pictures a second. On average. Peak traffic * 2.

Feed access - 500 million daily users * 10 access to feed. = 5 Billion requests for feed a day. 58k requests per second

Photo size = 2MB, we will have to store 50TB a day. Petabytes worth of data over time.

API design

Define what APIs are expected from the system...

upload image to S3, S3 handles massive scale effectively, streaming large data.

POST /image { imageS3Uri: ..., caption: ... }

GET /upload-url for presigned URL for S3

response {status: ..., message: ... }

For S3 we will use pre-signed URL, we get S3's speed and efficiency for free, while this is also secure.

GET /feed/

infer userId from headers (auth token) for these 2

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...

1. Photo storage. We need massively scalable object stores for photos, so S3 is a good option. It also offers great upload and download speeds. Offers all that we need to deal with large files at scale
2. Users Table: user_id, name, email, etc.
   1. we can use SQL for this since we can keep a strict schema, and have consistent writes for when user's change their information. There shouldn't be very heavy reads into this table as people aren't updating their profiles and info that much
3. Posts Table: post_id, caption, user_id, s3_url, etc.
   1. I'm going to choose a NoSQL DB for this like DDB. It is massively scalable and reliable, and it is subject to heavy reads so I want the scalability. As well, this will be a massive table so I want the managed partitioning. As well, the eventual consistency feature of DDB will work just fine as its okay for updates to be a little bit delayed.
4. GraphQL for follower relationships. This fits the use case perfectly.
5. Each user will have their own feed cache. We will write posts data to the cache upon write to help take of pressure from reads.
6. CDN for celebrity images.

High-level design

You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...

We should have multiple types of clients. Lets start with mobile and web.

for posting, we should have a posting service. behind AWS API gateway (handles authentication). it has 2 APIs, upload-url and image-post. The web services behind a load balancer will interact with S3, as well as writing to DB. As well, we should have a fanout on write for users with less than 100k followers, where we update their feed cache.

For the feed flow, the feed service should get feed information from the cache. Or for each high follower person, read their most recent posts and merge it with the cache.

Any events that require notification will hit the notification service.

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

Image upload flow deep dive.

POST /image { imageS3Uri: ..., caption: ... }

GET /upload-url for presigned URL for S3

we want to leverage S3 for scalable uploads and takes pressure off our service if we don't need to handle large files like images and short videos (good for cost too).

To upload to S3 securely from the client, we require a presigned URL. We create a GET /upload-url endpoint where users request a url to upload to s3. This is then used on the client side to directly upload to s3 through AWS client. Upon upload, get the S3 path. This will then be used for our POST /image API. So that we are only storing the URI.

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

Get Feed Flow

Users will sign in, go to their homepage, and /feed will be called. The user will already be authenticated so we will have their user_id. Find their feed cache.

For following that has more than the configured 100k, we will need to go into the feed cache and db for that.

Stitch the posts together in reverse chronological order. The images should still be fetched and rendered on the client side to reduce load on the server. Feed should provide a presigned url.

For feed, I'm considering also adding a general cache on top, partitioned and all so that content can be fetched even faster for celebrities. For both the DDB and Redis cache we use for the feed, we will use user_id as the partition key. We need to do this for fast retrieval. With accessing the main feed DB, we are querying per user (the flow is look up what users you are following that are celebrities, then get their most recent posts)

We can get into a hot key problem. we could break their data into different shards. To do that we can choose a composite key of (user_id + timestamp?)

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

I think i mention tradeoffs in my above writeup.

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

We are paritioning and replicating our posts data. We use eventual consistency.

We are using direct access to S3 to avoid bottlenecks with our webservers and api

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?