System requirements

Functional:

users are able to post photos, videos and texts

Non-Functional:

availability

low latency

scalabity

secure

monitor and analytics

Capacity estimation

1. in total 100 million users, DAU is 20 million
2. each user post 2 posts per day, 40 million posts per day, 40m/86K, 4000/8.6 -> 500 RPS, peak 2.5K RPS
3. each post 1K text + 5M photo, 40 * 10^6 * 5MB -> 200TB per day, 200T*365->73 Pega Bytes, 10 years -> 1EB
4. separate the photos and text, 1K * 40 * 10^6 = 40GB for texts per day, 40G*365 -> 15T per year
5. web server to handle the post and read, # of cores = 0.5 seconds * 2.5K -> 1.3K cores for posting, 100 subscribers per user
6. for reading, 2.5K * 100 RPS of reading, # of cores = 0.1 seconds * 250K -> 25K cores,

API design

post v1/userid/newpost {

text:

media:

timestamp:

}

get v1/userid/postid

Database design

1. use S3 to store the photos and videos
2. use NoSQL to store the post texts?
3. will use Cache, queue for processing?

High-level design

Make a post

1. user -> LB -> webserver
2. web server
3. webserver-> Media Service (compress the pictures, or videos) -> S3,
4. webserver -> NoSQL DB to save the post
5. web server -> notification service
6. Web server -> fanout service
7. Fan out service,
8. Fanout service gets all subscriber user info
9. generate news feed for each of the user
10. push the news feed to each subscriber's message queue
11. Message queue was consumed by fan out worker
12. Fan out worker will start to put the news feed or the post in each user's post cache

Read a post

1. user -> LB -> webserver
2. web server get the post details from postservice
3. Post service
4. post service connect to user post cache, retrieve the media from S3 and the post details from post DB to converge the post and show it to the user
5. post service also connect to user DB to get the publisher's detailed info

Request flows

Make a post

1. user -> LB -> webserver
2. web server
3. webserver-> Media Service (compress the pictures, or videos) -> S3,
4. webserver -> NoSQL DB to save the post
5. web server -> notification service
6. Web server -> fanout service
7. Fan out service,
8. Fanout service gets all subscriber user info
9. generate news feed for each of the user
10. push the news feed to each subscriber's message queue
11. Message queue was consumed by fan out worker
12. Fan out worker will start to put the news feed or the post in each user's post cache

Read a post

1. user -> LB -> webserver
2. web server get the post details from postservice
3. Post service
4. post service connect to user post cache, retrieve the media from S3 and the post details from post DB to converge the post and show it to the user
5. post service also connect to user DB to get the publisher's detailed info

Detailed component design

1. Regarding the user post cache, we pre calculate the post list for each user, so that when they pull the new posts, they will get the posts faster. For popular publishers like celebrities who have millions of subscribers, pre-generate the post cache will take a long time, we would rather the user to poll the posts, the post will be shown to the user by pulling.
2. When a publisher post a post with media, we need to compress the media and upload it to a storage, use a reference link to that storage and save the post in postDB. This is to save post DB space, also it is cheaper. Compress the photo or video will also save bandwidth
3. the user post cache should only be something simple, like userid, postid.
4. We can add CDN to store the photos and videos, so that when load the photo, post service dont have to read from s3 storage. It is directly loaded from the CDN. Set photos and videos to have TTL on CDN.

Trade offs/Tech choices

Failure scenarios/bottlenecks

load the media can be very time consuming, use CDN and cache will help a lot

Future improvements

NA