Loading...
GET /feed/me/ - fetches feed for the active user
POST /feed/me?content={content}&auth_token={auth_token} content is the data the user wants to post; authToken is used to check user's permissions. body={media:JSON} on body we can send any other media like images or videos for that post
POST /post/{postId}/react/{reactType}&auth_token={auth_token} used by users for reacting to a post
POST /post/{postId}/comment/{comment}&auth_token={auth_token} used by users to leave comments to a post
POST /post/{postId}/share&auth_token={auth_token} used by users to share a post
POST /notify/user/{userId} body:{notificationMessage:string} used to notify a user of a new post or interactions
see feed flow:
post and interact with posts flow:
in case of system failures, we have 5 levels of caching:
if any of the db fails, we temporarily use its cache, in case of fanout workers failures, then the message still remains in the fanout queue and can be picked up by other workers. in case the queue fails, it persists its data on disk and when it comes back up it will still have its data
when a post is viral, we will add its content to the hot cache, and we will seperate it from the normal cache so it can be accessed multiple times, if needed we can also store it multiple times accross different caches and regions so it will let more people access it (randomly access one of the caches, eg 4 caches).
users save their preferences for each post using usersService, whenever we compute their feed (after a new post is created), we first check if they have any blocked/muted accounts and filter them out.
to minimize cold caches, we will use a fanout on write approach for active users and whenever a new post is created we update the subscribers newsfeed caches too. this way we will have less cold caches and the system will be able to answer them quicker. when rebuilding the cache, the system checks the users graph db and the connections there, it then fetches the users from db and the posts and shows chronologically with a bias on the connectivity between users. when the ranking service is unavailable, we can use 40% chronological order and 60% linkings between users.
during region outages, a nearby region could take up the users even though their responses will be delayed. this region will need to scale out to manage this many requests
during spikes, users must fetch data from multiple shards (maybe the people they follow are spread accross multiple shards) so we will do parallel fetches to retrieve the data quicker
failover mechanism for replicas: for each shard we can have a master and 2 slaves, in case the master db fails then a slave will take its place until its back up
we also need to implement user level rate limiting, we could use token bucket and let each user post maximum 3 posts in a second
DNS/Anycast Failover Process: Route53 setup with health checks every 10 seconds on each region's /health endpoint. After 3 failed checks (30 sec), region marked unhealthy.
When cache is down or cold, multiple requests for same content hit DB simultaneously. Coalescing prevents this:
First request for post X triggers DB fetch. All subsequent requests for post X in next 50ms don't make new DB calls — they wait for first request's result. When first request completes, all waiting requests receive same data.
Implementation: in-memory map tracks "in-flight" requests by key. Before DB call, check if request already in flight. If yes, attach to existing promise. If no, create new promise and register it.