Loading...
2 billion DAU
100 to 1 read to write ratio
post 1KB
ratio of images 0.6 and average size of 1MB
ratio of videos 0.1 and average size of 10MB
Posts uploaded a day: 10^9
Posts read a day: 100* 10^9
Storage capacity:
Posts: 10^9*1KB = 1TB a day -> 365TB a year -> 1.8PB 5 years
Images: 0.6*10^9*1MB = 0.6PB a day -> 219PB a year -> 1EB 5 years
Images: 0.1*10^9*10MB = 1PB a day -> 365PB a year -> 1.8EB 5 years
Total Storage: 585PB a year
Bandwidth
2PB a day -> 2PB/(24*60*60) * 8 = 184Gbps
Server for peak load:
assuming dau can be peak load and each server can handle 64000 requests
2*10^9/64000 = 31250
POST /api/v1/post
{
auth_token,
content,
image_ref,
video_ref,
}
Get /api/v1/posts/{post_id}
{
content,
image_ref,
video_ref,
}
Get /api/v1/newsfeed?limit={x}&since={last_check_time}
({user_id} is fetched from auth_token)
[post_ids]
POST /api/v1/posts/{post_id}/like
POST /api/v1/posts/{post_id}/unlike
POST /api/v1/posts/{post_id}/comment
{
comment_content
}
GET /api/v1/posts/{post_id}/comments
[comments_id]
GET /api/v1/comments/{comment_id}
{
sender_id,
comment_content
}
POST /api/v1/follow
{
followee_id
}
(follower_id is fetched from auth_token)
users table SQL
{
pk: user_id,
email,
creation_time,
name,
}
user_followers table SQL
{
pk: follower_id, followee_id
}
Both tables will be sharded by user_id (follower_id in user_followers table). We need sql server since we need to have acid properties like unique email and user_id, foreign_key for user_followers follower_id and followee_id, and relational queries to find followers.
The rest of the data will be stored in lsm-tree DB like dynamodb or cassandra since we need to support high throughput.
post_id is user_id#sequential_id
posts table NOSQL
{
pk: user_id,
sk: sequential_id,
creation_time,
content,
video_ref,
image_ref
}
like table NOSQL
{
pk: post_id,
user_id,
creation_time,
}
comments table NOSQL
{
pk: post_id,
sk: comment_id (sequential id)
user_id,
creation_time,
content,
video_ref,
image_ref
}
The posts table will be sharded by the user_id. The In case of a hot shard for some users, we can further shard them by using sub paritions in the pk like USER123#0 and USER123#1.
The other tables will be shared by their primary key, which is post_id
Graph DB to query friends of friends
Nodes:
{user_id, name}
Edges:
{
label: follows,
}
Blob Storage:
Images and videos
Load balancer is used to distribute users requests
api gateway routes requests to the right service and also performs authentication and authorization to requests and also rate limitining if users performs too many requests
post service is responsible to storing and fetching posts, likes, and comments.
Kafka events store is used to store user's events like posting, adding a like or a comment. Kafka is used in order to decouple these events processing from the critical path and enable low latency for users' requests.
newsfeed generator is generating newsfeed by ranking posts using the ranking service and then storing the posts in an in-memory db like redis so users can quickly fetch newsfeeds. the newsfeed generator is used both in response when users upload new posts and for celebrities posts newsfeed generator service will rank and push celebrities posts while their followers request a newsfeed.
The newsfeed service will fetch newsfeed from the newsfeed cache and by generating on the fly ranking for posts from celebrities.
the notification service will notify users on viral posts.
Posts requests:
like/comment requests:
Fecthing a newsfeed:
Post service
the post servers are stateless; therefore, we can autoscale them based on cpu load. They store posts/likes/comments in a nosql db like (like dynamodb) that supports high throughput write. The DB is replicated for fault tolerence and auto-scales based on user_id sharding. in case of a user with a lot of posts and like/comments traffic, we can futher shard by adding sub partitions to the user_id.
Newsfeed service
The servers are stateless so they could support scaling based on cpu load. The posts for non-celebrities are already ranked in the newsfeed cache so they will be fetched quickly to support low latency for requests. Celebrities posts will be ranked on the fly. The newsfeed cache will be sharded by user_id, and since newsfeed requester only requests posts for himself, no join in needed between the shards. The posts will be stored in redis in a sorted set data structure based on their rank score, so we can query for top N ranked posts quickly in log(n) time.
The Kafka event store is used since it's a highly available and scalable component. It's used to decouple the user's events processing from the user requests processing. Thus, making our system more available and fault tolerent to partial failures to outages and network partitions in the newsfeed generator and notification services. The kafka event store will be partitioned based on topic per user.
Explain any trade offs you have made and why you made certain tech choices...
Try to discuss as many failure scenarios/bottlenecks as possible.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?