Loading...
List functional requirements for the system (Ask the chat bot for hints if stuck.)...
List non-functional requirements for the system...
Estimate the scale of the system you are going to design...
Comment Size = 100 KB max
Total number of users = 1M
Total Num Comments posted per day (nested and top level) = 1M approx
Total storage required per day = 100KB * 1M = 100GB
Total storage required for 6 months (before moving to tape backup) = 18 TB
Num users accessing the system = 10K per second approx
Num users viewing a thread or commenting on it concurrently = 100 max for popular threads or posts
This is a read-heavy system as users mostly read/follow threads and writes are less frequent.
Define what APIs are expected from the system...
POST https://
{
content: byte array
comment/postId: id of parent comment or post
}
Response: 201 Created with Comment Id
PUT https://
{
content: new content
}
Response: 201
DELETE https://
Response: 200 OK
GET https://
Response: JSON array of
{
commentId:
content:
from: userId
reactions:
}
same API and response as previous API
POST https://
{
post/commentId:
reactionId:
}
Response: 201 OK
Similar API
Elastic Search APIs
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
Given the scale of data, NoSQL DB is suitable for the system.
Posts and Comments
=================
Posts and comments are stored in a GraphDB. Every Post and Comment is a node in the graph. The main post is the root node. Replies and nested replies are children. This is shown in one of the diagrams.
Each node has the following information:
Search
=======
ElasticSearch DB is used for efficient text-based search on the comments.
Users DB
========
This can be an SQL DB with user information.
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
Content Distribution Network (CDN): As most requests are read-only, CDN can help reduce traffic to our system by servicing several requests directly.
Rate Limiter: Used to throttle the number of concurrent incoming requests to counter DDoS attacks.
Authorization: Used to authorize a user if they can post/reply/view a comment. It uses Users DB to access user data.
Comment/Post Mgmt: Service which accepts and managers the comments and posts.
Cache: LRU cache which caches the most recent or frequently accessed posts.
Graph DB: Stores the posts and comments.
Notification System: Notifies users when someone adds a comment to their post or posts they follow.
Search: Search service allowing full-text search for posts. It uses Elastic Search DB for efficient search
Analytics: Provides analytics data over popular posts, trending topics, user preferences etc.
Monitoring: Monitors posts and comments for sensitive or violating content.
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
Content Distribution Network (CDN): As most requests are read-only, CDN can help reduce traffic to our system by servicing several requests directly.
Rate Limiter: Used to throttle the number of concurrent incoming requests to counter DDoS attacks.
Authorization: Used to authorize a user if they can post/reply/view a comment. It uses Users DB to access user data.
Comment/Post Mgmt: Service which accepts and managers the comments and posts.
Cache: LRU cache which caches the most recent or frequently accessed posts.
Graph DB: Stores the posts and comments.
Search: Search service allowing full-text search for posts. It uses Elastic Search DB for efficient search
Analytics: Provides analytics data over popular posts, trending topics, user preferences etc.
Monitoring: Monitors posts and comments for sensitive or violating content.
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
As shown in the High Level Diagram, every post and comment is a node in the GraphDB and is connected to its immediate parent. The main post is the root node of a conversation thread. Each node has the following information:
When a user posts a comment(nested or top level), a node with the above information is inserted in the graph.
The Comment Management System traverses the graph of the post to find the comment, retrieves and edits it.
The Comment Management System traverses the graph of the post to find the comment to delete. All replies to that comment (all children of that node) are deleted.
Fetch all children of the Post node. Use pagination for posts with many comments.
Elastic Search is used for efficient search and retrieval.
Explain any trade offs you have made and why you made certain tech choices...
Try to discuss as many failure scenarios/bottlenecks as possible.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?