Loading...
Graceful degradation: When any component fails, the system falls back to a less optimal but functional state rather than returning errors.
Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...
Post writes: 2B DAU with roughly 1.8 posts per day per active poster. That is 3.6B new posts per day, or approximately 42K writes per second. During peak hours (evenings in major time zones), expect 2-3x spikes to 100K+ writes per second.
Read-to-write ratio: Approximately 3:1 on raw request count, but the asymmetry is deeper. Each write produces one database insert. Each read must consider posts from the user's entire social graph (hundreds of sources). Without precomputation, each read would require hundreds of database queries.
Post storage: Average post size is roughly 2KB (text, metadata, media references, actual media lives on CDN). At 3.6B posts per day, that is 7.2TB of new post data daily, or roughly 2.6PB per year. Cassandra handles this with horizontal scaling.
Design a personalized news feed system that aggregates posts from friends, pages, and groups, ranks them using ML models, and serves results in under 200ms to billions of users.
You open Facebook and see 20 posts. Behind those 20, the system evaluated 1,500 candidates from your 500 friends and 200 followed pages, ranked them with an ML model, and returned results in under 200ms. The real challenge is not showing posts. It is deciding which 1,500 candidates to even consider when you follow 700 sources and each posted multiple times today.
Personalized feed: Aggregate posts from friends, followed pages, and joined groups into a single ranked feed. Each user sees a different ordering based on their interaction history.
Post creation: Users publish text, images, videos, and link previews. Each post is immediately visible to the author and eventually appears in followers' feeds.
Reactions and comments: Users engage with posts through reactions (like, love, angry) and threaded comments. Engagement signals feed the ranking model.
Real-time updates: Online users receive notifications when new posts are available. The client decides when to fetch rather than auto-refreshing, which prevents jarring feed shifts mid-scroll.
Feed ranking: Posts are ordered by predicted relevance, not reverse chronology. The ranking model weighs recency, engagement probability, content type preference, and relationship strength.
Low latency: Sub-200ms feed rendering. Users perceive anything above 300ms as sluggish. The feed is the first thing they see on app open.
Massive scale: 2B+ daily active users. The system must handle 100K+ feed reads per second and 40K+ post writes per second simultaneously.
Eventual consistency: A 3-5 second delay between posting and appearing in followers' feeds is acceptable. Strong consistency across 2B users would require distributed transactions that are impractical at this scale.
High availability: 99.99% uptime. A blank feed is never acceptable, degraded feeds (stale, unranked) are always preferable to no feed.
Graceful degradation: When any component fails, the system falls back to a less optimal but functional state rather than returning errors.
Focus on feed generation, fan-out strategy, and ranking pipeline, these are the core engineering challenges. Exclude ad serving, Marketplace, Stories, and Messenger integration. Mention these as separate systems that inject content into the feed pipeline but are not part of it.
The read-write asymmetry reveals the architecture. 2B daily active users generating 3.6B posts per day means 42K writes per second. But 10B+ feed fetches per day means 120K reads per second. Each read touches hundreds of sources. This asymmetry is why precomputed feeds exist. You cannot merge 500 sources in real time at 120K requests per second.
Post writes: 2B DAU with roughly 1.8 posts per day per active poster. That is 3.6B new posts per day, or approximately 42K writes per second. During peak hours (evenings in major time zones), expect 2-3x spikes to 100K+ writes per second.
Feed reads: Each user opens their feed 5+ times per day, each time fetching 20-50 posts across multiple scroll loads. That is 10B+ feed requests per day, or roughly 120K reads per second sustained, with peaks at 300K+.
Read-to-write ratio: Approximately 3:1 on raw request count, but the asymmetry is deeper. Each write produces one database insert. Each read must consider posts from the user's entire social graph (hundreds of sources). Without precomputation, each read would require hundreds of database queries.
Post storage: Average post size is roughly 2KB (text, metadata, media references, actual media lives on CDN). At 3.6B posts per day, that is 7.2TB of new post data daily, or roughly 2.6PB per year. Cassandra handles this with horizontal scaling.
Feed cache: Each active user's feed cache stores 200 post IDs with ranking scores. At 8 bytes per post ID plus 8 bytes for the score, that is roughly 3.2KB per user. For 500M highly active users, the feed cache totals approximately 1.6TB in Redis, split across a cluster.
Social graph: 2B users averaging 500 friend and follow edges each. At 16 bytes per edge (two user IDs), the raw graph is roughly 16TB. Indexed and replicated, the social graph service requires roughly 50TB of storage.
A single feed response returning 20 post IDs with scores is roughly 500 bytes. At 120K requests per second, that is 60MB/s for feed metadata alone. Full post content (fetched separately with media URLs) adds significantly more, but CDN handles the heavy media delivery.
Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...
Two communication patterns serve this system. REST handles discrete actions (creating posts, fetching feeds, adding reactions. WebSocket handles continuous streams) notifying online users that new posts are available. The feed endpoint is the most latency-sensitive API in the entire system because it is the first thing every user sees.
GET /v1/feed?cursor=&limit=20 returns a page of ranked post IDs with precomputed scores. The cursor is an opaque token encoding the last seen position in the ranked list, enabling stable pagination even as new posts arrive. Each response includes a next_cursor for the subsequent page.
The response contains post IDs and ranking scores, not full post objects. The client uses these IDs to fetch full content in a batched second request. This two-step pattern lets the feed service focus entirely on ranking speed while content delivery leverages CDN caching.
POST /v1/posts accepts text content, media references (pre-uploaded to CDN via a separate media upload endpoint), visibility settings (public, friends-only, custom list), and an optional location tag. Returns the created post with its ID immediately. The author sees their post before fan-out begins.
WebSocket connection at /v1/ws/feed receives lightweight notifications: "N new posts available since your last fetch." The server never pushes full post content over the WebSocket. It only signals availability. The client displays a "new posts" banner and fetches when the user taps it.
This pattern avoids two problems: wasted bandwidth pushing content the user may never scroll to see, and jarring feed rearrangement while the user is reading.
Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.
Post Service: Validates the post, writes it to Cassandra, and publishes a "new post" event to Kafka. The author receives a success response here, before any fan-out happens. This service is stateless and horizontally scalable.
Fan-out Service: Consumes post events from Kafka. For each new post, reads the author's follower list from the social graph cache, then writes the post ID into each follower's Redis feed cache. This is the most write-intensive service in the system.
Notification Service: Listens to the same Kafka topic. For each new post, checks which followers are currently online (connected via WebSocket) and pushes a lightweight "new post available" notification. Never sends the full post. Just a signal.
Feed Service: The entry point for feed requests. Reads the user's precomputed feed from Redis, merges in posts from any "pull sources" (celebrity accounts followed by this user), and passes the combined candidate list to the Ranking Service.
Ranking Service: An ML inference service that scores each (user, post) pair based on predicted engagement. Returns the top N posts sorted by relevance score. This service is stateless and GPU-accelerated for throughput.
Step 1: User submits a post. The client sends POST /v1/posts to the API Gateway, which authenticates the request and routes to the Post Service.
Step 2: Post Service validates the content (text length, media references, visibility settings) and writes the post to Cassandra. The post is now durable.
Step 3: Post Service returns a success response to the client. The author sees their post immediately. Everything below happens asynchronously.
Step 4: Post Service publishes a "new post" event to Kafka with the post ID, author ID, and timestamp.
Step 5: Fan-out Service consumes the event and retrieves the author's follower list from the social graph cache in Redis.
Step 6: Fan-out Service checks the author's follower count. If below the threshold (roughly 5K), it executes fan-out on write: ZADD the post ID into each follower's Redis feed sorted set, batched at 1,000 followers per batch.
Step 7: If the author's follower count exceeds the threshold, the Fan-out Service skips fan-out and marks the author as a "pull source." Their posts will be fetched at read time instead.
Step 8: Notification Service (consuming the same Kafka topic) pushes a WebSocket notification to online followers.
Step 1: User opens the app. The client sends GET /v1/feed to the API Gateway, which routes to the Feed Service.
Step 2: Feed Service reads the user's Redis sorted set (ZREVRANGE) to get the precomputed feed, up to 200 post IDs with ranking scores from fan-out on write.
Step 3: Feed Service checks if the user follows any "pull sources" (celebrities). For each pull source, it queries Cassandra for their recent posts (last 24-48 hours) that have not yet been scored.
Step 4: The precomputed feed and pulled celebrity posts are merged into a single candidate list of roughly 500-1,500 posts.
Step 5: The candidate list is sent to the Ranking Service, which scores each (user, post) pair using the ML model. The model considers recency, predicted engagement, content type preference, and relationship strength.
Step 6: Ranking Service returns the top 20 posts sorted by score, along with a cursor for pagination. The Feed Service returns this to the client.
Step 7: The client uses the post IDs to fetch full post content (text, media URLs, reaction counts) from a content cache or Cassandra, then renders the feed.
Define the data model. Identify the main entities, their attributes, and relationships. Consider the choice of database type (SQL vs NoSQL) and justify your decision based on access patterns...
The social graph stores relationships: who follows whom, who is friends with whom, which groups a user belongs to. These relationships are bidirectional (friendships) or unidirectional (follows, page likes).
users: user_id (PK), username, display_name, created_at
friendships: user_id_1, user_id_2, status, created_at. Composite PK (user_id_1, user_id_2), indexed both directions
follows: follower_id, followee_id, created_at. PK (follower_id, followee_id), index on followee_id
PostgreSQL fits because the social graph is read-heavy (check friendship status, list all friends) with infrequent writes (follow/unfollow events). Relational indexing supports fast lookups in both directions: "who does user X follow?" and "who follows user X?"
Posts are write-heavy (42K/sec) and access patterns are predictable: fetch recent posts by a specific author (for fan-out on read) or fetch a specific post by ID (for content retrieval).
posts: author_id (partition key), created_at (clustering key DESC),
post_id, content_type, text, media_urls, visibility
posts_by_id: post_id (partition key),
author_id, content_type, text, media_urls, visibility, created_at
Partitioning by author_id means all of a user's posts live on the same Cassandra node, enabling efficient range queries: "give me the last 50 posts by user X." The clustering key sorts by created_at descending, so the most recent posts come first without an explicit sort.
Redis serves three purposes in this system, each with a different data structure.
Feed cache: Sorted set per user. The member is a post_id, the score is the ranking score. ZREVRANGE returns the top-N posts in O(log N + M) time. When a new post fans out, ZADD inserts it with its score. ZREMRANGEBYRANK trims the set to keep only the top 200 entries.
Social graph cache: Set per user containing friend and follow IDs. Used by the Fan-out Service to quickly retrieve the follower list without querying PostgreSQL on every post. Invalidated on follow/unfollow events.
Post metadata cache: Hash per post_id storing frequently accessed fields (author name, content type, text preview). Reduces Cassandra reads for content that appears in thousands of feeds simultaneously.
Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.
"How do you handle a celebrity with 50 million followers posting?" This is the question that separates a good answer from a great one. Pure fan-out on write means 50 million Redis writes per post. Pure fan-out on read means every reader merges from hundreds of sources at request time. Neither extreme works alone at Facebook's scale.
Hybrid Fan-out: Push for normal users (below 5K followers), pull for celebrities (above 5K followers)
Fan-out on write for users with fewer than 5,000 followers. This covers 99% of all posts. The Fan-out Service reads the follower list, and for each follower, executes ZADD to insert the post ID into their Redis sorted set with the timestamp as the initial score. For a user with 500 followers, this takes roughly 50ms (500 Redis operations). For 5,000 followers, roughly 500ms. The cost scales linearly with follower count.
Fan-out on read for users with more than 5,000 followers, celebrities, public figures, popular pages. When a reader opens their feed, the Feed Service detects that the reader follows one or more pull sources. It queries Cassandra for each pull source's recent posts (partitioned by author_id, so this is a single-partition range scan per celebrity). These pulled posts are merged with the precomputed feed before ranking.
The threshold is tunable. Facebook reportedly uses roughly 5,000. The right number balances two costs: below the threshold, you pay in write amplification (more Redis ZADD operations). Above the threshold, you pay in read latency (more Cassandra queries at feed read time). The threshold should be set where the write cost of fan-out exceeds the amortized read cost of pulling, which depends on the ratio of "posts per day" to "feed reads per day" for that user's followers.
The Fan-out Service processes each post event as follows:
If the Fan-out Service crashes mid-batch, it restarts from the last unacknowledged Kafka offset. Some followers may receive duplicate ZADD operations for the same post. But ZADD on an existing member is idempotent (it updates the score), so duplicates are harmless.
Two-Stage Ranking: Filter 1,500 candidates to 500, then ML-score to top 20
Ranking happens in two stages to balance quality with latency.
Stage 1: Lightweight filter: Remove candidates that should not be ranked: posts older than 7 days, posts the user has already seen (tracked in a Bloom filter or Redis set), posts from blocked users, posts that violate content policies. This reduces the candidate pool from roughly 1,500 to roughly 500. The filter runs in under 5ms because it uses only lookups, no ML inference.
Stage 2: ML scoring: For each remaining candidate, the model predicts engagement probability: P(like), P(comment), P(share), P(time_spent > 30s). These probabilities are combined with learned weights into a single relevance score. The model uses features from three sources: user features (demographics, recent interaction history), post features (content type, author, age, current engagement counts), and edge features (relationship strength between user and author, interaction frequency).
The top 20 posts by score are returned with a cursor encoding the 20th post's score for pagination. Subsequent pages re-rank the remaining candidates, so later pages still reflect relevance rather than just recency.
Key Insight
The hybrid threshold is not just about write cost. It is about cache freshness. A user with 500 followers gets their post into all follower feeds within 1 second via fan-out on write. A celebrity with 50M followers would take minutes to fan out completely. By which time, the post is already stale for early followers who already refreshed their feed. Pulling celebrity posts at read time actually provides fresher results than pushing would.
Common Pitfall
Interviewers expect concrete numbers. Fan-out on write for 500 followers: 500 Redis ZADD operations at roughly 10K ops/sec per shard, completing in about 50ms. Fan-out for 50M followers: 50M ZADD operations across the Redis cluster, taking roughly 83 minutes at 10K ops/sec per shard. This is why the threshold exists. The cost difference between 500 and 50M followers is not linear in practice, it is