Loading...
List functional requirements for the system (Ask the chat bot for hints if stuck.)...
Users should be able to create tweets
Users should be able to search their own tweets
Users should be able to search texts from other users (does not have to be their followers)
List non-functional requirements for the system...
Availability - The application should be highly available given that twitter is a global service
Latency - We want our search latency to be relatively low. It does not need to be immediate but should be
Consistency - We are a social media application where users do not need to see the most immediate information. We will aim for eventual consistency
Scalability - Our application should be scalable
Estimate the scale of the system you are going to design...
Assuming we have 100M daily active users.
Each user makes at least 1 tweet per day
Each tweet is on average 1KB
Each day, we expect to use 1KB * 100M = 100GB of storage per day for tweets.
For a year we will need 40TB of storage per year. Assuming 3x replications and indexing we will need about 150TB for a year
For qps
Assuming at peak, 1/5 of our user base will be using our search. 20M search requests / 24hrs / 60m / 60 s = 20M / 100,000 = 200 q/s (read)
For write TPS we are expecting 100M / 100,000 = 1000 q/s (write)
Given the capacity estimation we will need a database that will can scale horizontally and handle higher throughput.
A NoSQL database will filfull these requirements better than a SQL database. We can use a column db such as Cassandra
Define what APIs are expected from the system...
POST createTweet(user_id, text) -> returns status code
POST follow(user_id, follow_user_id) -> returns status code
GET search(user_id, text)
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
tweets table
{
tweet_id (primary key)
user_id, (references who wrote the tweet)
message, (text of the message)
timestamp, (when the tweet was created)
hashtags, (hashtag mentions of this tweet)
media_links (media links)
}
as our database grows, we can use user_id for sharding
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
Explain any trade offs you have made and why you made certain tech choices...
Try to discuss as many failure scenarios/bottlenecks as possible.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?