Loading...
List functional requirements for the system (Ask the chat bot for hints if stuck.)...
1. A User should be able to post comments.
2. A User should be able to reply to comments.
3. A User should be able to see a category of comments.
4. A User should be able to see a list of comments under different categories
List non-functional requirements for the system...
1. High Availability. We should provide a good reliable experience for a client when viewing/commenting on comments.
2. Scailability. The system should be able to handle the data load needed to get the job done and to support adding more users.
3. Consistency. We should ensure discussions and conversations are organized in a consistent manner.
Estimate the scale of the system you are going to design...
Assuming:
200M users
50M DAU
10 posts/reads a day
Each post can have 500 words and 1 picture. the 500 words is about 1000 bytes and the bigture is about 10000 bytes
each post is 11000 bytes
total is 10 * 11000 = 110000 bytes
Transactions per second:
5 * 10^6 / 10^5 = 1 * 10^1 = 10
Max database load per day:
50 * 10^6 * (110 * 10^3 ) / 10^5
5500 * 10^4
55 * 10^6 = 55Million bytes * 2 (data replication
MAX database load per second = 110 million / 10 = 11 millon bytes
TPS = 11 million / 10 = 1.1 million bytes per second
Define what APIs are expected from the system...
1.
https://getCategories (GET)
(timestamp, user_id)
Get all categories
2.
https://getCategory/category_Id (GET)
(timestamp, user_id, category_id)
Get a specific category
3.
https://handlePost/category_id/post_id/message_id (POST)
(timestamp, user_id, category_id, post_id, message_id, content, media)
Create a message for a comment on a category
4.
https://handlePost/category_id/post_id (POST)
(timestamp, user_id, category_id, post_id, content, media)
Create a post for a category
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
user_table
user_id: string
timestamp: string
index: user_id
categories_table:
category_id: string
content: string
category_title: string
timestamp: string
user_id: string FK
message_id: string FK
index: category_id
message_table
content: string
media_url: string
user_id: string FK
category_id: string FK
message_id: string FK
timestamp: string
index: message_id
Object_storage:
media_url: content
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
1. If an image is what the client is trying to load, the request goes to the CDN MEdia Cache.
2. Requests go through the API Gateway which handles authentication
3. If a user is requesting to view a category, categories, or a post the Content Cache will return the results if the Time to live property is still valid otherwise the content cache forwards the request to the post service.
4. If a user comments or creates a post the request goes to the post service. The post service communicates with the coordinator who routes the request to the proper database using consistent hashing
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
Caching
The CDN Media Cache is essentially used for caching images and videos. This significanly lowers the latency since the CDN server is near the client and media in general is expensive. The CDN Server updates periodically to ensure it has the latest content. There's a Time To Live property also on all content of around one hour, but it's configurable so it can adapt to how we see fit. The CDN server also speeds up our entire system since it helps alleviate many requests that could overwhelm the database. The CDN server is it's own service and follows our micro service architecture which helps us scale since we can scale that service independently.
The Content Cache is used for read requests. This too is updated periodically and we are okay with eventual consistency. This means it's possible it may not have all the updates immediately but are okay with the trade off since it significantly lowers latency since requests don't have to go to the database. We allow certain items to update more frequently if for example we pick up many write and read requests going to a certain id. We use a web hook on the database level to listen for certain times when we want to update the cache immediately.
Coordinator
The coorindator is service such as Zoo Keeper which manages requests going to our database. The database has one primary write DB and two follower DBs. If one of the database servers goes down the coordinator uses total order broadcast and a write ahead log to ensure only one new leader is promoted and to ensure the failed server updates when it comes back online. We have sharded the database by the user_id and the coordinator uses consistent hashing to ensure the data load is spread out evenly among our nodes. The coordinator also helps make sure our system keeps running smoothly by tracking the databases via heart beats. The coordinator meets our Scailability non-functional requirement because it can also promote more servers if needed.
Explain any trade offs you have made and why you made certain tech choices...
1.
Our database uses a Nosql database over SQL because our users that create posts are vital to our existence, without them we don't have a company. The Nosql database we use is better for writes because it uses a LSM Trees and SS Tables. The LSM tree stores write information in a memory table and when it gets full it offloads it to a SS Table. Both the LSM Trees and SS Tables use a data structure that allows us to do Binary Search to achieve logarithmic time for many actions. We accept that reads are slower because it may have to search both the SS Table and the LSM Tree, but we overcome this fault on our side by using caches.
2.
We use eventual consistency over Strong Consistency such as Linearizability. This is because we want to strike a balance between both the reads and writes. We are not responsible for critical information such as financial data and accept that some users may have a slight delay getting some updates on some posts and what not. We use write ahead logs to ensure that no update is lost and all nodes will get the updates sooner or later...usually within a couple of minutes.
Try to discuss as many failure scenarios/bottlenecks as possible.
There's a possibility that some posts may get overloaded with requests. We won't be able to handle a situation where all of a sudden tons of users try to comment on one comment since we only have one write database.
There's a possibility that the caches could be too out of date for some users and we may need to "tune" the refresh request of the cache.
We don't have multiple nodes of each component in this design, but there should be so we're completely fault tolerant.
We should implement a maintenance server.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?
We can connect the coordinator to multiple nodes and have that handle the promotion or demotion of different components. This would ensure we can scale on every micro service.
We can use a special shard/database to handle certain posts that have many more messages than other. This is essentially the celebrity problem.
The API gateway can be enhanced to handle maintainability along with the coordinator, which would allow us to track all components and catch errors.
The caching could be improved by tracking the last time the post was updated, and we could only cache old posts that have not been updated in a while and not cache messages that are being updated frequently. This would strike a balance and provide a good scalable design.