Loading...
100M / (24 * 3600) = 1157QPS
RW ratio: 10:1 (read tweets/post and follow and add tweet in favor folder)
1157 * 5 QPS in peak traffic
POST /v1/tweet/post
content
s3Url:
}
response:{
statusCode
message
id
}
GET /v1/tweet/{tweet_id}
response:{
content
s3_url
created_at
updated_at
}
GET /v1/tweet/all?cursor={cursor}&limit=20
response:{statusCode, message tweets:[],pageSize}
PUT /v1/tweet/{tweet_id}
requests:{
content
s3URL
}
DELETE /v1/tweet/{tweet_id}
response:{statusCode,message}
POST /v1/follow/following
requests{
followee_id
}
response:{statusCode,message}
GET /v1/follow/followers //get all followers
response:{
statusCode
message
followers:[]
}
GET /v1/follow/followees // get all followee
response:{
statusCode
message
followee:[]
}
GET /v1/follow/{user_id} //get all tweets under a user
response:{
statusCode
message
tweets:[]
}
DELETE /v1/follow/{follower_id}
response:{statusCode,message}
POST /v1/favor/add // add tweet in favorate folder
request:{tweet_id}, response:{statusCode,message,id}
DELETE /v1/favor/remove //remove tweet in favorate folder
response:{statusCode,message}
So high level design is
API Gateway:
rate limiter
authorization
routing
middleware
Post Service
post tweets to write in DB which we choose DynamoDB, Post service write Post under user's post list
If post tweets including image, should also put image in S3 and update CDN(cloud front),
Friend Service
viewing followers and followees, following or unfollow someone
Feed Service
it has two part of logics. Fan out read and Fan out write
Fan out write is for normal user post, it will go to friend service to get all followees id, write in a feedIndexTable which used for fetch tweets for all users, user_id will be partition key in DynamoDB which can make it faster to fetch each user's top 20 tweets. Also we have LRU redis clusters to reduce the query traffic to hit to DB
Fanout read is for those celebrity post, instead of write their posts to feedIndexTable, we have a precompute table to collect celebrity posts, and every time users will go to that table to fetch tweets post by celebrity they followed.
Feed service write will through message queue, and push to async workers to write to DynamoDB, in order to handle concurrent write requests
Choose DynamoDB is because high availablity requirement, and PK and GSI can make query by index faster.
FeedIndexTable
id
post_id
user_id
created_at
updated_at
Friend
id
user_id
followers:[]
followeees:[]
created_at
updated_at
POST // it also works as state machine table
id
user_id
content
s3Url
status: pending/in_progress/completed/failed
idepoency_key
created_by
followed_by
Favorute
id
user_id
favorates:[post_id1, post_id2...]
updated_at
created_at
precomputeTable //for celebrity
id
post_id
user_id
created_at
updated_at
First we have hotkey read issues in our LRU redis clusters, what if some of hot posts might be read most frequently, first we expect celebrities post might have higher possibilities as hot post, but some of post from normal people also possible to become hot post, in first case hot key is predictable, we can use some of replica nodes in redis particularly handling hot keys which post by celebrities, once plenty of read traffics come, local cache can handle amounts of them , the rest of them hit on hot key replica nodes. in order to make leader node of redis to know which replicas to routing, we need to use consistent hash to calculate partition id of replica nodes. and use zookeeper of etcd to store topology, once any replica node(especially hot key nodes) down, control plane compoent can detect it and auto recalculate consistent hash ring to re assign partition ids. I think. for DB also the same, leader and replicas machisms, so we also need to think about what if leader node down, we need to ensure replica node with latest updated replication logs warm up. once leader node down, a new leader election can pick replica with latest update replication logs can become new leader with less latency. However, we also need to prevent split brain issue if old leader node come back and still think its leader.
Tweets are scored by recency and recent engagement velocity to select the top ~500 for the precomputed feed cache" would cover this.