System requirements

Functional:

• Post a tweet
• View tweets
• Follow other users tweet
• Put tweets in favorites

Non-Functional:

• Scalable: the system should be able to handle large traffic, especially during peaks of traffic (for instance when a famous user post a tweet)
• Latency must remain acceptable (requests should be answered in < 1s)
• High availability, at leat 99.9%

Capacity estimation

• 500M daily active users
• Users post between 1 to 10 tweets each day (~2.5B tweet posted daily)
• Users views on average 20 tweets per day (10B tweets viewed daily)
• 12.5B requests each day

API design

GET /tweet/{id}

POST /tweet

POST /tweet/{id}/favorite

GET /feed

GET /users/{id}

GET /users/{id}/tweets

POST /users/{id}/follow

POST /users/{id}/unfollow

Database design

Tweet

• [PK] ID INT
• [FK] UserID INT
• TextContent VARCHAR
• CreatedAt DATETIME
• UpdatedAt DATETIME

Users

• [PK] ID INT
• Login VARCHAR
• Email VARCHAR
• Password VARCHAR

Feed

• [FK] UserId INT
• [FK] TweetID INT

Followers

• [FK] UserID INT
• [FK] FollowerUserID INT

We should use a partitioning strategy to avoid hot keys (famous users can make partitions unbalanced)

Adding a salt to the userid can help prevent this

High-level design

Requests are handled by different services so we can scale them independently

• Tweet service
• Feed service
• Users service

Feeds are generated asynchronously when tweets are posted. They are then cached as a single JSON

Frequent viewed tweets are also cached

Request flows

Detailed component design

Feed processing system:

• Each newly posted tweet creates an item in the tweet queue
• Feed processing nodes dequeue tweets
• Followers of the tweet author are fetched from the followers database
• Each follower feed is updated in the database and cached as a single gzipped JSON (with the tweets content for maximum performance)

Trade offs/Tech choices

• We can use eventual consistency to improve scaling
• Tracking followers can be done using a graph database so we can have a better understanding of relationships between users even if they are not directly connected

Failure scenarios/bottlenecks

• A famous users (lot of followers) posting a tweet can be a bottleneck because there will be numerous feeds to update. In our design the feeds are updated asynchronously to handle this case

Future improvements

• Enchance tweets with the possibility to include medias
• Add Monitoring and analysis systems