Requirements

Functional Requirements:

List the key functional requirements for the system (Ask the AI for hints if stuck)...

1. Post a tweet
2. home feed, read timeline of tweets from followers
3. Like/dislike a tweet
5. Search
7. out of scope
8. Follow and unfollow user
9. User auth

Non-Functional Requirements:

List the key non-functional requirements (performance, scalability, reliability, etc.)...

1. Low latency: users should be able to read with less than 50msec.
2. Read heavy system 50:1 read write ratio
3. System should be high available 99.999% uptime.
4. Eventual consistency is ok. Feed can take some time to appear.
5. Safetey: user accounts are authenticated, contents are safe.

Capacity Estimation

Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...

1. 2 tweets per user per day. 500M DAU = 1B tweets per day ( write) = 1B/100000 secs = 10000 writes/second
2. Average 100 tweets read per day, 500M*100 = 50B read per day = 5M reads/second.
3. Each tweet = 1KB, 1B tweets = 1B*1KB = 1TB per day, 365 TB per year.

API Design

Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...

API desing:

1. post /api/v1/tweets
2. {userid, tweet text, title, time}
3. -> tweetid
4. get /api/v1/tweets/{userid}/feed?
5. {lasttweetid}&limit=20
6. post /ap1/v1/tweets/{tweetid}/like

High-Level Design

Client talks to API gateway. For write flow, API gatewy forwards request to Tweet service. Tweet service writes the tweet to MongoDB ( document database,), we need to follow fanout strategy here since it is read heavy. After writing to DB tweet id is sent back to client. Also posted onto message queue, tweetid, tweet , userid etc. for each of the tweet, safetcheck is performed. once passed, feed generator gets the tweet, and user id, and find all users that follows this user, and feed that in to the redis data set. For read flow, API gateway reads the redis data set sorted based on ranking, and sent back to user with cursor pagination. If feed is not found, it is read from tweet DB and filled in redist data set compared to 50 msec, it might take 500 msecond.

Database Design

TweetDB

• tweet_id (primary key, auto-generated).
• author_id (indexed for author lookups).
• timestamp (indexed for chronological queries).
• content (up to 280 characters).
• media_url (link to image or video in object storage).
• like_count (periodically flushed from sharded counter).
• hashtags (array of hashtag strings extracted at write time).

User Collection

• user_id (primary key).
• email (unique index).
• name, display_name.
• follower_count, following_count (maintained via async counter updates)

FollowDB

• follower_id + followed_id (compound index).
• timestamp.

Hashtag Collection

• hashtag string (primary key).
• tweet_ids (array of tweet IDs containing this hashtag).

Detailed Component Design

Fan-Out on Write

When a user tweets, the Home Feed Generator looks up their follower list and inserts the tweet (with its computed score) into every follower's Redis Sorted Set. For an average user with 200 followers, this means 200 cache writes per tweet. Across 1B tweets per day, this produces 200B fan-out writes per day.

This works because the writes are asynchronous and distributed across many Redis servers. The tweeting user does not wait for fan-out to complete. The Message Queue absorbs traffic bursts. And 200 writes per tweet is a small, predictable cost.

Tweet Database Sharding (by Tweet ID)

Sharding the tweet database by tweet ID distributes writes evenly regardless of author popularity. A celebrity and an average user both generate tweets with randomly distributed IDs, so no single shard becomes a hot spot.

The trade-off is that querying "all tweets by user X" requires scatter-gather across all shards. But this is acceptable because hashtag lookups (the more common query pattern) are fast because the Hashtag document stores tweet IDs directly, and the system can route each ID to its specific shard.

Feed Cache Sharding (by User ID)

Sharding the Redis feed cache by user ID means every feed read hits exactly one shard. The Home Feed Service hashes the user ID, finds the shard, and retrieves the Sorted Set. Each user's feed is roughly the same size (20 tweets), so the load distributes evenly.

The trade-off is that the Home Feed Generator must write to many shards during fan-out. A tweet from a user with 1,000 followers requires writes to potentially hundreds of different cache shards. But writes are async and batched, making this manageable.