System requirements
Functional:
List functional requirements for the system (Ask the chat bot for hints if stuck.)...
Be able to post to twitter
Be able to read other peoples tweets
Be able to see other users tweets
Be able to like other peoples tweets
Non-Functional:
List non-functional requirements for the system...
Scaleable for many people tweeting at one time
High Reads
Available, uptime and disaster recovery
Monitoring and logging
Follow other users
Capacity estimation
Estimate the scale of the system you are going to design...
100 million users
Each user makes 2 tweets a day
Each user reads 50 tweets a day
Each user likes 10 tweets a day
So we assume we can have a system that doesn't require many writes in comparison to reads
API design
Define what APIs are expected from the system...
All of the API's should be restful, for authentication we will use OAuth2 and generate a token from a micro service that would be the authorization server
One API to view all a users tweets GET
One Api to view all followed users tweets GET
One api to view a specific tweet GET
One api to favorite and unfavorite a tweet POST
One api to create a tweet POST
One api to Follow other users POST
Database design
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
We could use a NOSQL graph DB for users following each other to keep track of that,
and then for each tweet we can have a SQL database, one table for a users tweets and another table for the tweets they have liked, using a primary key on the tweet to connect both tables together.
Account's of the users info could be also held in a separate table which would just be user info. So 3 tables in total
We should cache the most popular tweets and also the most popular user's and their tweets
High-level design
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
I want a microservice design,
A user management microservice
a tweet creating and viewing microservice
a following microservice
a feed generation service
And a notification service which will be based on publish subscribe method
Request flows
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
It would start with a user logging onto our website and would be then sent to the authentication service to go login.
Then once logged in there homepage would call the feed generation service that would give them a feed based on all of the user follows
After that they will have the choice to like other peoples tweets on their feed which woudl call a microservice, or instead create there own tweet which would call a microservice for tweet creation
Once a user creates a tweet then all of their followers would get a notification as we will have a notification service get all upon creation of any tweet
Detailed component design
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
So for the tweet creation micro service it will scale horizontally if we need more due to load. We will keep a load balancer in front of it, one active load balancer and one passive so theres no single point of failure, then that load balancer will point to an API- gateway that will handle authentication and logging, which will finally point to a tweet creation microservice. For amazon I would leverage ECS. Part of our API-gateway will have rate limiting on it so the system can't be spammed or abused
For the feed generation microservice what we can do is that we can have the same loaderbalancer schema and api -gateway I spoke up for the tweet creation micrservcice but for this one what we can do is make it query for all followers latest tweets, we can just get that days tweets, from our sharded database that is partitioned based on day of tweet.
Then for global tweets from unfollowed users we can just grab them for the cache of our tweet creation microservie as that cache will hold some of the most popular tweets and recently created/polled for tweets
Trade offs/Tech choices
Explain any trade offs you have made and why you made certain tech choices...
consistency vs. Availability: Discuss the CAP theorem and how you've balanced consistency, availability, and partition tolerance in your design.
Cost vs. Performance: Analyze trade-offs between using expensive hardware vs. optimizing software for cost-efficiency.
Scalability vs. Complexity: Evaluate the balance between easily scalable solutions and the potential increase in system complexity.
Failure scenarios/bottlenecks
Try to discuss as many failure scenarios/bottlenecks as possible.
If one of our database shards fails then we will have an issue as it's shared on the date. The feed generation microservice is reliant on the tweet microservice.
If the notificaition microservice fails then we will not have a way to let our users know the latest tweets
I think a failure scenario would be what if a user deletes there account what happens to their tweets
Or also unliking a tweet
or also thinking about if we delete a tweet what happens to the like in the DB
Future improvements
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?