Loading...
Total Users - 2B
Users Data (metadata + name, email, gender etc) - 1KB/user
Total storage = 2 * 10^9 * 1KB = 2TB
Daily uploads = 1000 uploads per sec = 10^8 uploads in a day.
Each upload is 2MB on an average.
total storage in a day = 2MB * 10^8 = 200TB
Total storage in a monthy = 6PB
Total storagey in a year = 6PB * 12 = 72PB/year
Writes throughput - 1000 uploads/sec
Peak uploads - 10K uploads/sec (10x)
Write throughput for other activities (likes/comments) can be at roughly 10 likes/day and 5 comments/day (total 15 activities/day/user)
Total activities = 1B * 15 = 15B activities/day
150K activities/sec with peak at 450K activities/sec
Reads: A user on an average reads 5 feed requests (containing 20 posts each) -> 5B reads/day = 50K reads/sec with peak at 200K reads/sec
Cache size - 500 cached items/users at 1KB
Total size = 1KB * 1B * 500 = 500TB of hot feeds
Each user on an average follows 200 people. Total metadata = 200 * 2B * 200bytes = 80TB
We will have APIs for the following things
User Account Related
Create User Account
HEADER:
Idempotency-Key: uuid
Authorization: Bearer
Body: {
name: string,
email_address: string,
address: string
phone_number: string
...
}
Response:
{
user_id: uuid,
created_at: timestamp
}
Follow/Unfollow user
POST /user/follow/{user_id}
POST /user/unfollow/{user_id}
Response:
{
success: True
}
Uploads
POST /v1/upload
HEADER:
Idempotency-Key: uuid
Authorization: Bearer
Body:
{
caption: string
tags: string,
num_post_items: int [Number of photos/videos users wants to upload]
}
Response:
{
urls: [url1, url2, ...]
}
Response:
{
post_id: id
success: True
}
Feed
GET /v1/feed/{user_id}
Response:
Header:
Content: "application/json"
If-Not-Match: Key
Body:
{
user_id: uuid,
posts: [
{
caption: string
tags: string
likes: int,
urls: [url1, url2, ...]
comments: [{user_id, uuid, body: string, parent_id: None, timestamp: timestamp, likes: int}, ......]
}, ....
],
has_more: True,
next_cursor: uuid
}
Activity
Like/Dislike
POST /post/like/{post_id}
POST /user/dislike/{user_id}
Response: {user_id, uuid, post_id: uuid, like_count: int}
Comment
POST /post/{post_id}
Body: {user_id, uuid, comment: string}
Response: {
comments: [{user_id, uuid, body: string, parent_id: None, timestamp: timestamp, likes: int}, ......]
post_id: uuid
}
We will have different microservices.
User upload a photo:
Liking a post:
Comments:
Notification Service:
Feed Service:
Since we want to serve the feeds fast, we will divide the feed service into broadly 2 parts.
For users will less followers, the data will be read from Kafka and updated in Feeds DB / Redis.
For celebrities, who has millions of followers, we will keep the data separate
When an user calls the feed service -> we will aggregate the pre-built feed + celebrity posts (who user follows) -> rank the posts using ML model, which will include engangement, user closeness etc
Serve the feed to the user
Here, if ML ranking is slow, we will fallback to recency ranking based on timestamp
Follow/Unfollow
We will store the data when users follow/unfollow users. this will be used to build the feed of the user
We will have multi region replicas, so that the latency is low for users across the globe. It will also be used as a fallback, if there is an outage in any region
Cache Rebuild:
We will have cache rebuild service, which will keep reading the data from kafka, and keep the Redis Cluster updated.
In case of a cache miss, we will follow cache aside pattern and update the data in the Redis.
Users DB: Postgres
(
user_id: uuid, [PK]
name: string,
email_id: string,
....
)
Posts DB: Postgres
(
user_id: uuid, [FK]
post_id: uuid, [PK],
likes: int [denormalized],
caption: string,
tags: string
created_at: timestamp,
updated_at: timestamp
)
Index: (user_id, post_id) -> This is to show on users profile
Image Table:
(post_id: uuid, url: string)
Indexed by post_id
Comments DB: Postgres
(
post_id: uuid,
user_id: uuid,
comment_heirarchy: "/comment_id1/comment_id2..."
)
Indexed by post_id
Likes DB: Postgres
(
user_id: uuid,
post_id: uuid
timestamp
)
Indexed by (post_id, user_id)
Follows DB: Postgres
(
follower: uuid,
followee: uuid
timestamp
)
indexed by (follower, followee)
indexed by (followee, follower)
Feeds DB: Cassandra
(
user_id: uuid,
feed: [
[post_id, user_id, caption, url], ....
]
)
this is denormalized for easier retrival and cached in Redis
We cache the data in Redis as well
Photo/Video upload handling
Likes
Comments
Follow/Unfollow:
Feed Generation:
Failure Handling:
Cold Cache rebuild:
Use the same single-flight lock (SET lock:feed:{user_id}:{page} NX) for rebuild, not only expiry. Winner range-reads the Cassandra timeline, hydrates cards, writes page:feed:{user_id}:{page}, notifies waiters; losers wait ~30–50ms and retry — they never all fall through.
Scalability: