System requirements
Functional:
users able to share tweets
users can subscribe to other users
- tweets sent by these users will appear in the subscriber's feed list
- time order
- up to 1000
users can favorite tweets
- there's a place to check all favorited tweets
- up to 1000
tweets can include both media files and texts
users receive notifications when people they subscribe post tweets
Non-Functional:
support user worldwide
high scalability, reliability for a big scale
low latency
Capacity estimation
100 million users, 10% DAU -> 10 million
10 million tweets created per day -> 100 write qps, ~ 200 peak qps
Suppose on average users check tweets 10 times per day -> 1k read qps to retrieve tweets from users they subscribe.
one char 1 byte, ~300B, let's assume on average 100B, and 10% of tweets have media, with average 3MB -> 1GB text, 1TB media every day ->~400GB text, ~400TB media per year
API design
REST APIs
- retrieve a list of tweets (read)
request {
long requestUserId
// for pagination
long anchorId
int limit
boolean isFavorite
}
response {
list
long nextAnchorId
}
struct Tweet {
long id
String text
List
}
- subscribe/unsubscribe to user (write)
request {
long requestUserId
long followingUserId
// enum action, can be follow or unfollow
Action action
}
response {
boolean successful
// error messages if not successful. Used by clients to deliver user-facing messages
Set
}
- post tweets (write)
request {
long requestUserId
String text
List
}
response {
boolean successful
// error messages if not successful. Used by clients to deliver user-facing messages. examples like reaching daily limit
Set
long tweetId
}
- favorite tweets (write)
request {
long requestUserId
long tweetId
}
response {
boolean successful
// error messages if not successful. Used by clients to deliver user-facing messages.
Set
}
Database design
due to the high write and read qps and big data volume, we will use NoSQL db for the following tables. Cassandra is a good choice
Tweet {
long id,
long userId,
boolean isDeleted,
Timestamp postedAt,
List
String text
}
UserConnections {
long followerUserId
long followeeUserId
Timestamp followedAt
}
FavoriteTweet {
long tweetId
long followerUserId
Timestamp favoritedAt
}
In addition to this, to power efficient read path (users see list of tweets by their followers), we can build a search index (like elastic search) for Tweet table. It can support quicker read and customized ordering and pagination search of the tweets
High-level design
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
Request flows
user query list of tweets:
sequenceDiagram
Client->>Server: Request list of tweets
Server->>UserRelationshipService: Query followers of user
UserRelationshipService->>UserConnection: Query UserConnection table
UserConnection-->>UserRelationshipService: Return list of user IDs
UserRelationshipService-->>Server: Return list of user IDs
Server->>TweetService: Query tweets for user IDs
TweetService->>TweetElasticSearchService: Query tweets
TweetElasticSearchService-->>TweetService: Return list of tweets
TweetService-->>Server: Return list of tweets
Server-->>Client: Return list of tweets
Client-->>CDN: download media files
user query favorite tweets
sequenceDiagram
Client->>Server: Request list of favorite tweets
Server->>FavoriteTweetTable: Read with user ID
FavoriteTweetTable-->>Server: Return list of tweet IDs
Server->>TweetService: Query tweets using tweet IDs
TweetService->>TweetElasticSearchService: Query tweets
TweetElasticSearchService-->>TweetService: Return list of tweets
TweetService-->>Server: Return list of tweets
Server-->>Client: Return list of favorite tweets
Client-->>CDN: download media files
user posts tweets
sequenceDiagram
Client->>CDN: Upload media
CDN-->>Client: Return media URLs
Client->>Server: Send tweet contents + media URLs
Server->>TweetService: Write to Tweet table
TweetService->>ElasticSearch: Sync data to index
ElasticSearch-->>TweetService: Confirmation of sync
TweetService-->>Server: Confirmation of tweet creation
Server-->>Client: Return confirmation of tweet posted
Detailed component design
API gateway:
to prevent malicious user/bad traffic, and also reinforce the per day limit, we will add rate limit to API gateway.
Trade offs/Tech choices
Explain any trade offs you have made and why you made certain tech choices...
Failure scenarios/bottlenecks
Try to discuss as many failure scenarios/bottlenecks as possible.
Future improvements
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?