Loading...
2 billion DAU
100 to 1 read to write ratio
post 1KB
ratio of images 0.6 and average size of 1MB
ratio of videos 0.1 and average size of 10MB
Posts uploaded a day: 10^9
Posts read a day: 100* 10^9
Storage capacity:
Posts: 10^9*1KB = 1TB a day -> 365TB a year -> 1.8PB 5 years
Images: 0.6*10^9*1MB = 0.6PB a day -> 219PB a year -> 1EB 5 years
Images: 0.1*10^9*10MB = 1PB a day -> 365PB a year -> 1.8EB 5 years
Total Storage: 585PB a year
Bandwidth
2PB a day -> 2PB/(24*60*60) * 8 = 184Gbps
Server for peak load:
assuming dau can be peak load and each server can handle 64000 requests
2*10^9/64000 = 31250
POST /api/v1/post
{
auth_token,
content,
image_ref,
video_ref,
}
Get /api/v1/posts/{post_id}
{
content,
image_ref,
video_ref,
}
Get /api/v1/newsfeed?limit={x}&since={last_check_time}
({user_id} is fetched from auth_token)
[post_ids]
POST /api/v1/posts/{post_id}/like
POST /api/v1/posts/{post_id}/unlike
POST /api/v1/posts/{post_id}/comment
{
comment_content
}
GET /api/v1/posts/{post_id}/comments
[comments_id]
GET /api/v1/comments/{comment_id}
{
sender_id,
comment_content
}
POST /api/v1/follow
{
followee_id
}
(follower_id is fetched from auth_token)
users table SQL
{
pk: user_id,
email,
creation_time,
name,
}
user_followers table SQL
{
pk: follower_id, followee_id
}
Both tables will be sharded by user_id (follower_id in user_followers table). We need sql server since we need to have acid properties like unique email and user_id, foreign_key for user_followers follower_id and followee_id, and relational queries to find followers.
The rest of the data will be stored in lsm-tree DB like dynamodb or cassandra since we need to support high throughput.
posts table NOSQL
{
pk: post_id,
user_id
creation_time,
content,
video_ref,
image_ref
}
like table NOSQL
{
pk: post_id,
user_id,
creation_time,
}
comments table NOSQL
{
pk: post_id,
sk: comment_id (sequential id)
user_id,
creation_time,
content,
video_ref,
image_ref
}
The posts table will be sharded by the user_id. The In case of a hot shard for some users, we can further shard them by using sub paritions in the pk like USER123#0 and USER123#1.
The other tables will be shared by their primary key, which is post_id
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
Explain any trade offs you have made and why you made certain tech choices...
Try to discuss as many failure scenarios/bottlenecks as possible.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?