Loading...
MVP:
Users can comment on a post
Supports nested comment structure (10 layers)
Users can see real comment updates
Non MVP:
Comments deletion, updates
Read >> write
low latency > consistency
Scalability: able to handle large read traffic or hot post
Consistency: eventual latency is chosen in favor of real-time requirement
Performance: the real-time update latency should be low
Durability: comments should be stored securely and safely
100 million DAU
each user views 10 posts / day
1 billion post views / day
1000 0 00 000 / 100 000 => 10k qps for views
10 : 1 read to write => 1k qps for comments
1000 * 10 000 => 10 Million qps for real-time event updates
Storage:
500 bytes for comments
Num_of_comments_per_day = DAU * views_per_user * 0.1
= 100 000 000 * 10 * 0.1 = 100 000 000
50 000 000 000 bytes => 50 GB / day
post_comment(text, resource_id) -> returns comment_id
resource_id can be resource_type + resource_id
like topic_topic_id or comment_comment_id
POST /comments
{
text: yyy
resource_id: xxx
}
get_comment_tree(resource_id, depth, limit) -> returns a json blob of comments
depth is the depth of subtrees
limit is num of comments to return
GET /comments/{:resource_id}?depth={:depth}&limit={:limit}
{
post_id: xx
comments: [
comment1: {
type: comment
data: {
comment_id: xxxx
content: xxxx
children_comments: {
comment2 : {},
comment3: {}
}
},
comment4: {
type: more
data: [comment_ids...]
},
]
}
https://www.reddit.com/r/redditdev/comments/110z3ii/issues_retrieving_the_entire_comment_tree_on_a/
Comment Table
comment_id
post_id (partition key)
parent_id
is_deleted
timestamp
user_id
content
primary key comment_id + post_id
If using Postgres, we can use Recursive Queries or Ltree type to build the dependency.
We use ltree here as it has built-in query syntax and is designed for query tree structure.j
INSERT INTO Comments VALUES ('comment1.comment2....');
When user deleted a comment, we will not delete the comment in the backend, but instead we set is_deleted field to true, this way we will not impact the tree structure. When returning the result to client, we can return null or a flag so the client knows this comment has been deleted
Connection Table
FrontEnd_server_id
subscribed_post
timestamp
We need a shared connection table, so that any connection service can consume an update msg, look up the connection table and then broadcast to front servers
We can have poll and push modes.
In poll mode, we let client periodically send request to server to pull latest info.
In push mode, we maintain a communication channel between client and server, and server pushes updates to client whenever there is an update. We can use
Server sent event(SSE) to maintain this communication.
Comparing poll and push, for poll approach, the implementation might be easier. But it would be tricky to get the right poll frequency. If the poll interval is low, then it would increase the server load. If the poll interval is high, then it will not push the updates in real-time. So we will use push mode here. Apart from the benefit of real-time push, by we can also use that push communication channel to deliver other real-time data for ex. likes, emojis.
Display Post comments
when user starts to view a post, the post service will query the Post DB and Comment DB
to get Post and Comment information.
To support nested comments, we will return a comment tree in response, client will display nested comments based on the comment tree. To not overload the system, service will return the tree with default depth of 2. Rest comments will be hidden with "more" button. After click "more", the service will use ltree to load the children comments
Push real-time comments
Besides displaying the comments, we will connect the user with a front-end server. The communication method is SSE. The front server will have an in-memory db to store the mapping between client and interested post_id.
The front-end server will send a request to connection service and reports that now this server is subscribed to this post. The connection service will store the server_id and post_id information in a DB.
When a new comment is added, the post service will firstly store this comment in the comment DB, and then sends a request to connection service to broadcast the updates.
If we add more data centers, we can broacast the comment updates to connection service in different data centers, and if the connection DB shows some clients are interested in this update, it will distribute the updates otherwise it will ignore
Trade-off between consistency and latency
In current setup, the delivery is at-most-once delivery, and there is no guarantee that the message will be delivered to the client.
We can use Kafka to get the delivery guarantees. The problem is scaling. Kafka can handle hundreds of thousands of topics, up to millions. But that may still not fit into our use cases (each post can be a topic). So we need to figure out a way to shard the queues (which event goes to which queue). The front-end server needs to subscribe to all queues (Kafka is pull based model, you dont unsubscribe, just stop consuming). This is not scalable. Add more front-end server will not help either, because new front-end server will still subscribe to all queues
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
Explain any trade offs you have made and why you made certain tech choices...
Try to discuss as many failure scenarios/bottlenecks as possible.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?