Loading...
Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...
For this service, we need different APIs for each operation:
When users want to post a top level comment or a reply:
POST v1/create_comment_or_reply
{
user_id: UUID,
parent_comment_id: UUID
}
When users want to edit a comment:
PUT v1/edit_comment {
user_id: UUID,
comment_id: UUID
}
When users want to delete a comment:
DELETE v1/delete_comment {
user_id: UUID,
comment_id: UUID
}
When users want to upvote or downvote:
PUT v1/vote_comment {
user_id: UUID,
comment_id: UUID,
vote_type: int
}
When users want to sort the comments by either time or popularity, and view the comments:
GET v1/read_comments {
user_id: UUID,
comment_id: UUID,
sort_by: Enum
}
Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.
First, we need to create the data models for this service.
table comment {
comment_id: UUID,
parent_comment_id: UUID,
created_at: timestamp,
updated_at: timestamp
comment_text: jsonb,
}
table comment_vote_user: {
comment_id: UUID,
user_id: UUID,
vote_type: Enum,
vote_time: timestamp
}
table comment_vote_counts: {
comment_id: UUID,
vote_count: int
}
We will store the data in a relational database like postgres, for 2 reasons:
To speed up the massive read QPS, we will have a Redis caching layer on top of the database to serve repeated reads. We will also create read replicas for the database, to handle the read volume.
For any requests made, we will first go through load balancer for load balancing, and then API gateway for authentication.
Here's an overview of the few flows:
Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.
In this deep dive section, we will deep dive into several questions/tradeoffs:
How do we ensure consistency of comments in a distributed system?
In a distributed environment, if user A posts reply to a comment, and that comment is in the process of being propagated to all database replicas, then user C could see comment made by A in 1 second, then in the next second it disappears. This is not acceptable since users should see consistent experience.
What we will do here is to use client side vector clocks. Whenever server receives a request, it sends back a vector clock documenting the highest version this client has seen. In this client's subsequent requests, server will only send versions higher than this vector clock.
Compared to alternative approaches like qurom and Linearizability, this approach doesn't provide that high level of consistency. However, it has lower latency than both approaches. In our use case, we don't need the user to always see the latest version as long as they don't experience a "time travel" effect. We would trade consistency for better availability and lower latency.
Also, if users submit duplicate comments, we will reject the requests through the use of idempotent keys.
How do we make sure that each post has the right distributed count, used for popularity sorting?
To handle the large amount of vote/unvote events, we use redis cache to store the vote count per comment. If the table becomes too large, we might need to shard the tables by comment_id, and uses sharded counters to get the count.
Everytime a user votes/downvotes, we trigger a Kafka event, that later writes to the database. The tradeoff is availability here, if the redis crashes before events are pushed to kafka, or if kafka crashes, we won't have stable storage that gives the accurate count. But that is a reasonable tradeoff compared to writing directly to database for each vote/downvote, which will overwhelm the database and cause availability problem.
How do we ensure real time delivery of messages?
For reply events on Kafka, it gets processed by the external notification service, and gets pushed to client through a SSE.