Fan-out Strategy: Hybrid over Pure Push or Pure Pull
Failure scenarios/Bottlenecks
Redis Feed Cache Failure
Future improvements
Time Well Spent Optimization
Table of Contents
Overview
System requirements
Functional
Capacity estimation
Traffic
API design
Feed Retrieval
Database design
PostgreSQL : Social Graph
High-level design
Write Path
Request flows
Post Creation Flow
Detailed component design
The GATE: Hybrid Fan-out Strategy
Trade-offs/Tech choices
Fan-out Strategy: Hybrid over Pure Push or Pure Pull
Failure scenarios/Bottlenecks
Redis Feed Cache Failure
Future improvements
Time Well Spent Optimization
Overview
Design a personalized news feed system that aggregates posts from friends, pages, and groups, ranks them using ML models, and serves results in under 200ms to billions of users.
You open Facebook and see 20 posts. Behind those 20, the system evaluated 1,500 candidates from your 500 friends and 200 followed pages, ranked them with an ML model, and returned results in under 200ms. The real challenge is not showing posts. It is deciding which 1,500 candidates to even consider when you follow 700 sources and each posted multiple times today.
Functional
Personalized feed: Aggregate posts from friends, followed pages, and joined groups into a single ranked feed. Each user sees a different ordering based on their interaction history.
The read-write asymmetry reveals the architecture. 2B daily active users generating 3.6B posts per day means 42K writes per second. But 10B+ feed fetches per day means 120K reads per second. Each read touches hundreds of sources. This asymmetry is why precomputed feeds exist. You cannot merge 500 sources in real time at 120K requests per second.
Traffic
Post writes: 2B DAU with roughly 1.8 posts per day per active poster. That is 3.6B new posts per day, or approximately 42K writes per second. During peak hours (evenings in major time zones), expect 2-3x spikes to 100K+ writes per second.
Two communication patterns serve this system. REST handles discrete actions (creating posts, fetching feeds, adding reactions. WebSocket handles continuous streams) notifying online users that new posts are available. The feed endpoint is the most latency-sensitive API in the entire system because it is the first thing every user sees.
Feed Retrieval
GET /v1/feed?cursor=&limit=20 returns a page of ranked post IDs with precomputed scores. The cursor is an opaque token encoding the last seen position in the ranked list, enabling stable pagination even as new posts arrive. Each response includes a next_cursor for the subsequent page.
Three storage tiers matched to three access patterns. PostgreSQL for the social graph (read-heavy, relational joins for friend-of-friend queries). Cassandra for posts (write-heavy, partitioned by author for time-range scans). Redis for precomputed feeds (sub-millisecond reads, ephemeral data that can be regenerated).
PostgreSQL : Social Graph
The social graph stores relationships: who follows whom, who is friends with whom, which groups a user belongs to. These relationships are bidirectional (friendships) or unidirectional (follows, page likes).
Five services, one key insight: the write path and read path are completely separate. A post travels through the write path (Post Service to Fan-out Service to feed caches). A feed request travels through the read path (Feed Service to Redis to Ranking Service). They share only the storage layer. This separation lets each path scale independently.
High-Level Architecture: Post Service, Fan-out Service, Feed Service, Ranking Service, and storage layers
Two flows define this system. Post creation is write-heavy (42K/sec) with async fan-out that trades latency for throughput. Feed fetch is read-heavy (120K/sec) with a strict sub-200ms latency target. Understanding both flows reveals why the system is split into separate write and read paths.
Post Creation Flow: Write to Cassandra, publish to Kafka, fan-out to follower feeds
"How do you handle a celebrity with 50 million followers posting?" This is the question that separates a good answer from a great one. Pure fan-out on write means 50 million Redis writes per post. Pure fan-out on read means every reader merges from hundreds of sources at request time. Neither extreme works alone at Facebook's scale.
Hybrid Fan-out: Push for normal users (below 5K followers), pull for celebrities (above 5K followers)
Three cascading decisions shape this system: how to fan out (push vs pull vs hybrid), when to rank (write time vs read time), and what consistency guarantees to sacrifice. Each decision constrains the next.
Fan-out Strategy Comparison: Write-heavy push, read-heavy pull, and hybrid approach
Fan-out Strategy: Hybrid over Pure Push or Pure Pull
The feed must always render. A user opening Facebook to a blank screen is unacceptable. Every failure mode needs a degradation path that still shows content. Stale content, unranked content, or incomplete content is always better than nothing.
Redis Feed Cache Failure
A Redis shard goes down, losing feed caches for a fraction of users. The Feed Service detects the cache miss and falls back to regenerating the feed from Cassandra: query recent posts from all followed users, run the ranking model, populate a new Redis sorted set, and return the ranked feed. This fallback takes 1-2 seconds instead of 200ms. Redis Sentinel promotes a replica within 5-10 seconds, after which normal performance resumes. During the failover window, serve the regenerated feed and optionally cache it at the CDN edge for users in the same region.
The system as designed handles reliable feed delivery and ranking at scale. The next frontier involves smarter ranking, privacy, geographic optimization, and content quality.
Time Well Spent Optimization
Move beyond engagement prediction (likes, comments, shares) to user satisfaction optimization. Downrank clickbait that generates clicks but leaves users feeling worse. Incorporate user satisfaction surveys as a training signal, periodically ask "was your time on Facebook well spent today?" and use responses to adjust ranking weights. This aligns business metrics with user wellbeing rather than maximizing session time at any cost.