System requirements
Functional:
List functional requirements for the system (Ask the chat bot for hints if stuck.)...
Users need to be able to perform CRUD operations on their posts
Users need to be able to see other users posts
Users need to be able to interact with others posts
Non-Functional:
List non-functional requirements for the system...
System needs to be able to handle millions of users
System prioritizes availability over consistency
System needs to provide low latency
Capacity estimation
Estimate the scale of the system you are going to design...
Twitter has millions of users ~400million
Each user probably sees ~100 posts
5 posts per day
400 million * 100 reads= 400 000 000 00 reads
400 million * 5 writes = 200 000 000 0 writes
API design
Define what APIs are expected from the system...
createUser(userName, encryptedPW)
createPost(userID, post)
fetchPosts(userID)
Database design
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
Users will have a SQL DB. Then we can track groups/followings
Posts will have a NoSQL DB. There will be much more posts than users, need posts to be scalable.
Posts Schema
Post: {
userId: String
Date: Date
Content: {
}
}
High-level design
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
We will have a Client that requests connetions
A CDN for regional connection
Load Balancer
Fleet of servers
microservices: createUser, getPosts, createPosts
two databases, UsersDB and PostsDB
Redis PostCache
Request flows
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
We will have a client request a connection.
Because Twitter is worldwide platform it'll go through a CDN so it can stay within region for lower latency
After it goes through the CDN we will have a load balancer, using Round Robin to distribute the load. This way the fleet of servers will have an even distribution of load.
After this we will be using microservice architecture for each API. This way we can independently scale each service depending on load.
We will have a getPosts, createUser, createPost microservice.
getPosts and createPosts will point to the same Database, because we want to lower latency, we will have a Cache for most recently fetched posts. We are assuming a created post will be one of the posts that need quick fetching for followers to see, so we will have createPosts directly update the cache, and the cache will update the posts DB.
Detailed component design
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
Trade offs/Tech choices
Explain any trade offs you have made and why you made certain tech choices...
Wanted to prioritize latency over throughput. Users are impatient, we want to provide them with the quickest updates possible.
Prioritize availability over consistency. We want data to be quickly available so we can give users posts to view. Consistency is not as important because the worst case is users have to wait too long for data, not if a posts likes are not what they actually are.
Failure scenarios/bottlenecks
Try to discuss as many failure scenarios/bottlenecks as possible.
Some failures are people huge followings on their accounts. When they post millions of users will need to fetch their post which might overwhelm the cache/servers.
Future improvements
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?