Requirements


Functional Requirements:


  • Allow users to tweet messages up to 140 characters.
  • Enable users to follow other users.
  • Allow users to like tweets from other users.
  • Display tweets from followed users in the home feed.
  • Show top K popular tweets in the home feed based on likes and followers.



Non-Functional Requirements:


  • The tweets should be highly availbile
  • Read-heavy workload. Each user tweets about twice a day but reads roughly 100 tweets. With 500M DAU, this produces a read-to-write ratio near 50 to 1, which means the system must be optimized for reads above all else.
  • The user should read teh tweets instantly
  • The system should be scalable to supports tweets from any country
  • user should easily see the followers and following
  • The tweets user posted should be consistent eventually


API Design

Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...


  • POST /tweets with body containing the content. Creates a tweet. Returns the new tweet ID. Server-side parsing extracts hashtags and user mentions from the content.
  • POST /users/{user_id}/follow with body containing the target user ID. Creates a follow relationship. Idempotent: following a user you already follow returns success without duplicate entries.
  • POST /tweets/{tweet_id}/like. Records a like. Increments the like counter (via sharded counter for popular tweets).
  • GET /users/{user_id}/feed?cursor={last_tweet_id}&limit=20. Returns a JSON object containing an array of tweets and a next_cursor value. The cursor is the ID of the last tweet returned, so the next request picks up exactly where the previous one left off.





High-Level Design

Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.

Two paths define this architecture. The write path stores a tweet and fans it out to followers' feeds through a message queue. The read path retrieves a pre-computed feed from cache in under 10 milliseconds. Separating these paths lets you optimize each independently: writes go through async message queues while reads hit pre-computed caches.

Write Path

The client sends a tweet to the API Gateway. The gateway authenticates the request, applies rate limiting, and forwards it to the Tweet Service. The Tweet Service stores the tweet in MongoDB and publishes an event to the Message Queue. Three async consumers process the event independently: the Tweet Analyzer extracts hashtags and user mentions, the Content Analyzer checks for policy violations, and the Home Feed Generator computes a score and pushes the tweet into every follower's Redis Sorted Set.

Read Path

The client requests its home feed through the API Gateway. The Home Feed Service checks the user's Redis cache first. On a cache hit (the common case), it returns the pre-computed top-K tweets in under 10 milliseconds. On a cache miss, it falls back to MongoDB, queries for followed users' recent tweets, computes scores, populates the cache, and returns the results in 100 to 500 milliseconds.

CDN

Static content (profile images, uploaded media, video thumbnails) is served through a CDN. These assets are written once and read millions of times, making them ideal CDN candidates.

Level Expectations

Mid-level: explain the write vs read path separation and why async message queues decouple tweet creation from feed distribution.

Senior: identify the celebrity fan-out problem and propose a hybrid push/pull strategy.

Staff: quantify the feed cache at 5.12TB across 40 servers, discuss sharding the cache by user_id, and design the fallback path when the pre-generation pipeline lags behind real-time.

Request flows

Two flows define this system. The write flow shows how a single tweet reaches millions of feeds through fan-out. The read flow shows how pre-computation turns a complex ranking problem into a simple cache lookup. Together, they demonstrate why separating write and read paths makes this system tractable at scale.

Write Flow (tweet)

  1. The client sends POST /tweets with the tweet content to the API Gateway.
  2. The API Gateway authenticates the request, applies rate limiting, and forwards to the Tweet Service.
  3. The Tweet Service validates the content (280-character limit), generates a tweet ID, and stores the tweet in MongoDB.
  4. The Tweet Service publishes a "new tweet" event to the Message Queue.
  5. The Tweet Analyzer consumes the event, extracts hashtags and user mentions, and updates the Hashtag collection.
  6. The Content Analyzer consumes the event and checks for policy violations.
  7. The Home Feed Generator consumes the event, looks up the author's follower list, computes a score, and inserts the tweet into each follower's Redis Sorted Set.

Write Flow: tweet(), Client to API Gateway to Tweet Service to DB + Message Queue

Read Flow (home_feed)

  1. The client sends GET /users/{user_id}/feed to the API Gateway.
  2. The Home Feed Service receives the request and checks the user's Redis Sorted Set.
  3. Cache hit path (over 99 percent of requests). The service retrieves the top-K tweets by score from the Sorted Set, resolves tweet content from a tweet cache or MongoDB, and returns the response in under 10 milliseconds.
  4. Cache miss path. The service queries MongoDB for the user's followed accounts, fetches their recent tweets, computes scores, inserts results into the Redis Sorted Set (backfilling the cache), and returns results in 100 to 500 milliseconds.

Read Flow: home_feed(), Cache Hit vs DB Fallback Strategy

Common Pitfall

Never propose building the feed on every read request. With 50B feed reads per day, on-the-fly ranking would execute the scoring algorithm 50 billion times daily. Pre-computing feeds on write reduces this to 1B scoring operations (one per tweet, fanned out to followers). The 50x reduction in computation is what makes the system feasible.



Detailed Component Design

Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.


Three data stores, each chosen for a different access pattern. Tweets go to a document database for flexible schema and horizontal scaling. Feeds live in Redis for sub-millisecond reads. The follows graph uses a separate collection to support efficient bidirectional queries.

Tweet Collection (MongoDB)

  • tweet_id (primary key, auto-generated).
  • author_id (indexed for author lookups).
  • timestamp (indexed for chronological queries).
  • content (up to 280 characters).
  • media_url (link to image or video in object storage).
  • like_count (periodically flushed from sharded counter).
  • hashtags (array of hashtag strings extracted at write time).

User Collection

  • user_id (primary key).
  • email (unique index).
  • name, display_name.
  • follower_count, following_count (maintained via async counter updates).

Follows Collection

  • follower_id + followed_id (compound index).
  • timestamp.


Sharded Counter for Likes

When a celebrity's tweet receives millions of likes, a single like_count field becomes a hot spot. The solution is a sharded counter with 100 sub-counters stored in Redis, each handling 1/100th of the write traffic. A Counter Service randomly assigns each like to a sub-counter. Every 5 minutes, the service sums all sub-counters and flushes the total to the tweet's like_count field in MongoDB.



The read path and write path:

Write Path

The client sends a tweet to the API Gateway. The gateway authenticates the request, applies rate limiting, and forwards it to the Tweet Service. The Tweet Service stores the tweet in MongoDB and publishes an event to the Message Queue. Three async consumers process the event independently: the Tweet Analyzer extracts hashtags and user mentions, the Content Analyzer checks for policy violations, and the Home Feed Generator computes a score and pushes the tweet into every follower's Redis Sorted Set.


Read Path

The client requests its home feed through the API Gateway. The Home Feed Service checks the user's Redis cache first. On a cache hit (the common case), it returns the pre-computed top-K tweets in under 10 milliseconds. On a cache miss, it falls back to MongoDB, queries for followed users' recent tweets, computes scores, populates the cache, and returns the results in 100 to 500 milliseconds.

CDN

Static content (profile images, uploaded media, video thumbnails) is served through a CDN. These assets are written once and read millions of times, making them ideal CDN candidates.

Fan-out is THE core challenge of this system. For regular users (under 1,000 followers), push on write works perfectly, since 200 cache writes per tweet is cheap. But a celebrity with 10M followers triggers 10M cache writes from a single tweet, overwhelming the cache cluster. The hybrid approach (push for regular users, pull for celebrities at read time) is what separates a working Twitter from one that collapses under a celebrity's tweet.