Requirements

Functional Requirements:

• Allow users to submit posts to subreddits
• Enable users to upvote or downvote posts and comments.
• Support threaded discussions with nested replies indented hierarchically.
• Allow users to create new subreddits based on different topics.
• Allow users to browse posts within subreddits
• Allow user to comment on posts

Non-Functional Requirements:

• Highly Available - 99.9% uptime when user browsing posts
• Low latency for feed reads
• Eventual consistency for feed
• High read throughput for feed and votes
• Highly Scalable to support traffic even during peak time

Capacity Estimation

• 100M Users
• 20M DAU
• 5M posts/day
• 50M comments/day

Feed traffic

• Each DAU -> 5 requests -> 400M requests/day
• 4K average QPS
• Peak -> 4x - > 20K

Storage:

• 5M posts/day
• each post ~2Kb
• 10 gb /day
• 3.6 TB/year

API Design

Create Subreddit

POST /subreddits

Get subreddit feed

GET /subreddits/{id}/posts?sort=x&cursor=...

Subscribe

POST /subreddits/{id}/subscribe

Create Posts

POST /subreddits/{id}/post

Get comments

GET /posts/{id}/comments

post comments

GET /posts/{id}/comments

Vote

POST /posts/{id}/vode

req - {value}

Value -> 0,1,-1

0 = remove vote

1 = upvote

-1 = downvote

Home feed

GET /feed?cursor=...

High-Level Design

API Gateway

Acts as the entry point for all client requests.

Responsibilities:

• Authentication (if required)
• Rate limiting - Applied using token bucket per IP to prevent fraudulent or bot views
• Request routing

Load Balancer

Distributes traffic across multiple instances of all services to improve scalability and availability.

User Service

• Crud apis for user
• handles user subcribing to a subreddit apis
• Posts user event to kafka like view, click, subscribed etc

Post Service

• CRUD apis for post
• manages post metadata in redis => posts:123 -> {content} and uses this caceh for faster retreival of posts
• Publishes evetns like PostCreated to kafka

Comment service

• CRUD apis for comment
• Handles retriveal of comment tree
• Also manages comment tree cache in redis => comments:post123 and uses this cache for faster retriveal of comments
• Publishes CommentCreated Event to kafka

Subreddit Service

• CRUD apis for subreddit

Vote Service

• Handles upvote/downvotes
• Stores each vote in cassandra and publishjes relevate vote events like VoteCreated to kafka

Feed Service

• Handles feed generation for User home feed as well as subreddit home feed
• Uses redis to fetch top candidates + ranking srevice to rank and return top candidate
• Applies pagination via cursorId

Vote aggregator

• Consumes VoteCreated event and aggreates votes by post
• Flushes the aggregation periodically to update the post/comment score in cassandra, as well as updates the post metadata and comment in redis

Top posts refresher

• Consumes VoteCreated, CommentCreated events, aggregates those by post
• Flushes that periodically and calculates the hot score of posts and then updates redis sorted set of posts in redis

Affinity Pipeline

• Consumes evetns like Views, clicks, VoteCreated, CommentCreated and aggregates those by user and subreddit
• Flushes that periodically to calcualte user affinity to subreddits and then updates redis sorted set of user subreddit affinity

Redis

• Caches:
  • hot posts for each subreddit => hotPosts:subreddit123 -> [posts] => maintined by hot feed refresher
  • post metadata and comment tree => LFU
  • User - subreddit affinity => maintained by affinity pipeline

Kafka

• Event backbone
• Decoples asyc workloads needed for downstream activities like updating post vote scores, updating top posts cache and user subsreddit affinity

Database Design

Reddit is a massive, read-heavy system where most access patterns are predictable and can be modeled around partition keys. We prioritize horizontal scalability and high availability over relational joins and complex transactions so using NoSQL like cassandra

Users

id - partiton key

username

email

Subreddits

id - partiton key

name

description

createdBy

createdAt

User subscriptions

Id

userId - partiton key

subredditID

isActive

createdAt

modifiedAt

Posts

Id

Title

Description

media_url

subredditId - partition key

AuthorId- FK to user id

score

CreatedAt -clustering key

Votes

Id

UserId - clustering key

EntityId - Parition key

EntityType - Posts/Comments

value

CreatedAt

Comments

Id

UserId

PostId - partition key

comment

parentCommentId

Score

CreatedAt

Detailed Component Design

We will now deep dive into the following topics:

Feed Generation

Feed is one of most high traffic througput and most complex component of the design

How it works?

• Feed requests hits feed service
• Feed service will fetch users top subreddits from redis using tghe user subreddit affinity cache
• For top 10 subreddits, it will then fetch top 20 posts for each of them -> again from redis using hot posts cache
• We now have 200 candidates, will send those candidates ot ranking service.
• These 200 are stored in Redis with
• Ranking service will sort them by rank
• These sorted posts is stored in redis as short lived cache with TTL = 5min, so when user is scrolling and needs next set of posts, our service isntead of reranking or refetching usinse the same snspashot and returns it, when user does full refreshed then feed service will receive request with no snapshot hence a full feed refresh will happen.

• It will enrich the top 20 posts with its metadata again from redis and return response
• Uses cursor pagination. The cursor contains the last item's ranking score plus a deterministic tie-breaker such as post ID. Since the personalized feed is dynamically ranked, we can also associate the cursor with a feed snapshot/version to keep pagination stable while the user pages through the feed.

Comment Design

• Comments are stored flat:

Example:

The hierarchy comes from:

Why not store the entire tree?

A popular post can have millions of comments.

One giant nested document would cause:

• huge reads
• difficult updates
• contention
• poor pagination

Instead, fetch incrementally.

Comment Cache

Comments are caches as LFU so popular comments are cached

Cache:

• top comments
• popular reply branches

Redis reduces repeated reads from the comment DB.

How does hot posts by subreddit cache and user-subreddit cache stay Fresh?

-> Top posts refersher and Afiinity PIpelien services are reponsible to keep them fresh.

How it works?

• Top posts refresher:
  • Consumes events such as VoteCreated/VoteUpdated, CommetnCreated/CommentUpdated and aggregates those events by posts
  • Every few second these aggregation is flushed and a hot score is caulcaute for each post involved
  • if the score is higher than the lowest-ranked post in the redis sorte set of hot posts then the post is added and the lowest ranked post is removed from cache
• Affinity pipeline:
  • Follows same logic as above
  • Consumes events, aggregates them by user and subreddit
  • Flushes those events to calcualte a affinity score and then update teh redis sorted set of User-subreddit affinity

Failure Handling

Redis failure:

For post metdata and comments:

• Fallback to DB
• Then asynchronously rebuild cache.
• Redis is a cache, not the source of truth.

For user affinity and hot posts:

• Feed generation can fall back to the recently subscribed subscription list and top recent posts for those subreddits
• These cache will be then rebuld by top posts refresher and afiinity services.

Kafka failure

Use:

• replication
• retries
• consumer offsets
• dead-letter queue for poison messages

Because events are durable, consumers can resume.

Database failure

Use:

• replication
• partitioning
• multiple replicas
• automatic failover