Requirements

Functional Requirements:

• Allow users to tweet messages up to 140 characters.
• Enable users to follow other users.
• Allow users to like tweets from other users.
• Display tweets from followed users in the home feed.
• Show top K popular tweets in the home feed based on likes and followers.

Non-Functional Requirements:

• High Availability
• Scale to 100+ millions of users
• handle a high volime of tweets , llkes and tweets
• Low Latency
• Durability
• Media delivery via CDN
• Eventual Consistency is acceptable
• High throughput and Scalability

Capacity Estimation

Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...

MAU 300M

DAU 150M

Tweet per user per day 2

Media 10% of the tweet 1MB avg

Tweet Record size 300B

Read:Write 100:1

Peak Factor 2x Avg

Retention 5 years

Replication x3

peak factor 2x avg

Write QPS

Tweet/day = 150M users x 2 = 300M tweet per day

Write QPS = 300M / 86400

= 3500 QPS

Peak = 2x 3500 = 7000/s

avg followers = 200-300

timeline inserts = 3.5k x 200-300

= 0.7-1M/s avg

peak = 7k x 200-300 = 1.4-2M/s

READ QPS

Read QPS = 100 x writes

100 x 3500

350000 /s

Peak = 2x 350000

= 700000 /s

Tweet Storage

Per day = 300M X 30Byte

90GB /per day

per year = 80GB x 365

= 33TB

5 years = 33TB x 5

= 164TB

Media Storage

Media tweet = 10% x 300M

= 30M /day

per day = 30M x 1MB

30TB / per day

per year = 11PB

5 years = 55PB

Replication

Total = 164TB + 55PB

= 55.2 PB

55.2 PB x 3 = 165 PB

brandwith

Ingress

Text = 90GB / per day /86400 = 1MB

media 30TB/day / 86400 = 350MB/s

Egress

350k x 300B = 100 MB

media 105 x 350k/s x 1mB = 35GB

Cache

150M DAU x 800ids x 8B

= 1TB

Provision 2-3TB

cluster = 2-3TB / 128-256GB per node

10-20 nodes

Social Graph storage

edges = 300M users x 200 avg follows = 60B follow edges

edge record = 8B+8B+8B + overhead = 40B

follows = 60B x 40B = 2.4TB

x tables = 5TB

x 3 replication = 15TB

API Design

Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...

Tweet & Feed API

POST /v1/tweets

GET /v1/tweets/{tweet_id}

GET /v1/tweets/{tweet_id}/replies?cursor=...&limit=20

GET /v1/feed/home?user_id=...&cursor=...&limit=20

GET /v1/feed/public?cursor=...&limit=20

DELETE /v1/tweets/{tweet_id}/like

GET /v1/feed/home?cursor={last_tweet_id}&count=20

Social Graph

POST /v1/follows

DELETE /v1/follows/{follower_id}/{followee_id}

Comment / Engagement

POST /v1/tweets/{tweet_id}/replies

POST /v1/tweets/{tweet_id}/like

Search & Media

GET /v1/search?q=...&type=tweets|users&sort=latest|top&cursor=...

POST /v1/media

Admin / Monitoring

GET /v1/health

GET /v1/metrics

High-Level Design

Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.

Database Design

Define the data model. Identify the main entities, their attributes, and relationships. Consider the choice of database type (SQL vs NoSQL) and justify your decision based on access patterns...

users

user_id (PK, snowflake)

username (unique)

email (unique)

created_at

tweets

tweet_id (PK,snowflake Time-sortable )

user_id (author,FK-> users)

tezt (<= 140 chars)

media_id (nullable - object storage ref , never the. bytes)

like_count ( dennormalized , refreshed async from Redis)

created_at

Follows

follower_id composite PK (follower_id, followee_id

followee_id composite PK

created_at

Followers

followee_id Composite PK

follower_id Composite PK

created_at

Likes

user_id composite PK

tweet_id composite PK

created_at

Storage Engine

• Tweet + edges -> Cassandra: append only, write heavy load fits LSM storage, horizontal scale, tunable consistency matches the availability > consistency
• Users-> PostgreSQL : low volume, need unique constraints (username/email) and transactional signup.
• Live counters -> Redis , flushed async to a durable Cassandra counter table.
• Sharded PostgreSQL

Tweet Partitioning

• Shard key - user_id places via consistent hash
• Why not shard by tweet_id? Perfect write spread, but "all tweets by user X" becomes scatter-gather across every shard — and both the profile page and the celebrity read-merge need exactly that query.
• Within a partition cluster by tweet_id DESC
• Hot partition guard

Follows & Like Partitoning

• follows -> shard by follower_id
• followers -> shard by followee_id
• likes -> shard by user_id

One table per access pattern

Home Timeline

Hybrid :

• Regular users -> fan out on write
• Celebrities -> fan out on read
• Why Hybrid : Pure write fan out = 50M inserts for one celebrity tweet.
• The timeline is not a DB table

Timeline Cache

Redis Cluster

• One capped list per active user ~ ~1TB across 10-20 nodes
• TTL 30 days of inactivity
• Seperate Redis LRU pool for hot tweet objects

Detailed Component Design

Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.

Feed generation

Push ( fanout on write ) , on tweet workers insert the id into every follower redis timeline

Pull (fan out on read ) : build the feed at request time from the follow list , but very slow

So approach will be hybrid , push for normal users dont fan out celebrities .

Top K popular Tweets

Kafka event -> stream process (flink) keep windowed engagement counts Redis ZSet feed the service blends top K into the timeline

Like Count

A viral tweet melt a single counter key -> shard the counter increment a random shard , sum on read . The likes give idempotency async batch flush to the durable store. aysync batch flush to the durable store

Media Path

Presigned upload -> object storage -> async transcode/thumbnail serve via CDN .

Cold Start , Queue lag and top K sizing

Cold start / cache miss : a new user or evicted timeline 30 days TTL -> timeline service falls back to fan out on read. read the follower list from graph store pull each followee recent tweets , merge sort by tweet_id , serve the page then write the result back to redis

Queue Delay / outage : the durable write happen before kafka publish. the source of truth is never behind the queue . Fan out is atleast once + idompotent . Kafka lagging 30s just mean feed are 30s stale , feed catch up automatically once the queue is healthy.