Requirements

Functional Requirements:

• Allow users to post top-level comments and replies in a nested hierarchy.
• Enable users to edit or update their own comments and delete them; deletion may be restricted if the comment has replies.
• Provide real-time updates or notifications when someone replies to their comment or posts in a discussion they follow.
• Allow users to upvote or downvote comments to indicate quality or popularity; vote counts influence sorting.
• Enable sorting comments chronologically or by popularity.
• Enforce a maximum depth for nested replies.

Non-Functional Requirements:

• Needs to have low latency (Users should see theirs and others posted comments in 10ms-50ms)
• Needs to scale horizontally (We need to be able to add more servers when amount of users increase by millions or write requests increase by 10 million)
• Needs to be reliable (Needs to have an uptime of 99.99% and if service goes down, needs to only be down for at most 10 minutes)

Capacity Estimations:

• Assume we get about a million comments posted a day.
• Assume a comment is about 500 bytes
• 1,000,000 requests * 500 bytes = 500,000,000 bytes = 500 MB
• 500 MB * 365 days = 182,500 MB = 182 GB

API Design

1. path: /postComment
   1. request: { userId: string, postId: string, postContent: string }
   2. response: { commentId: string }
2. path: /replyToComment
   1. request: { userId: string, commentId: string, postContent: string }
   2. response: { commentId: string }
3. path: /editComment
   1. request: { userId: string, commentId: string, postContent: string }
   2. response: { commentId: string }
4. path: /deleteComment
   1. request: { userId: string, commentId: string }
   2. response: None
5. path: /voteComment
   1. request: { userId: string, commentId: string, vote: integer }
   2. response: { votes: integer }
6. path: /getComments/{postId}?sortby={date/popularity}
   1. response: { List[commentId: string] }

High-Level Design

1. We will first have the Client retrieve static content from the CDN for performance.
2. We will then have a Rate Limiter which will prevent people from spamming comments or spamming votes. We will limit the amount of comments they make to 5 a second and limit their vote to once per post.
3. The request will then go through the Load Balancer to handle even load distribution to the server.
4. The Server will send the request to the Comment Handling Service.
5. The Comment Handling Service will handle all of our APIs.
6. When the user calls the postComment or replyToComment API, we will send an SQS message with the commentId as the payload.
7. The SQS will be sent to the Push Notification Service by checking who owns the post or the comment the comment was posted to and they will be notified with a push notification.
8. When we call postComment, replyToComment, editComment, deleteComment, or voteComment, we will first write it to the Write Through Cache. This ensures that our data will always be consistent on read.
9. When calling voteComment, we will use Write Aggregation to batch vote counts together until we reach a certain amount of votes (ex: 100 votes) or a certain amount of time has passed (ex: 2-3 seconds).
10. We will have a cache replica to handle read only requests to the cache so that we can horizontally scale if the amount of read requests we get increases.
11. When we call getComments, we will first check the cache replica (read only) to see if the postId is stored there. If there is a cache miss, we will check the database. Depending on what the user is sorting by, we will sort it by their choice and return the comments with pagination.
12. The Write Through Cache will finally write it to the database for long term storage.
13. We can deploy the database to multiple zones to ensure that if the database goes down, we can access one in another zone that is still up.

Detailed Component Design

1. Comment Handling Service
   1. postComment will take the postId and store the postContent into the cache and database under the userId.
   2. replyToComment will take the a commentId and store the postContent into the cache and database under the userId. We will also send a payload to the SQS with the commentId and the names of the user the poster replied to and the poster of the original post. The SQS will trigger the Push Notification Service to send a push notification to the users.
   3. editComment will take a commentId and replace the postContent in the cache and database with the same commentId and create a new version into the database. It will also be stored under the userId. If we attempt to edit a comment that doesn't match the version on read, we will not update the comment.
   4. deleteComment will set the item to "isDeleted" in the database. This will prevent issues with nested comments. On the frontend, the comment will read as "This comment was deleted."
   5. voteComment will take a 1 or -1 and will send that request to the Write Through Cache (Redis) so we can perform batching and atomic operations. The voted user will also be stored in the database so that they cannot vote on the same post again.
   6. getComments will first read from our cache replica to see if we can fetch comments from there. If there is a cache miss, we will check the database. We will return it in sorted order based on the users choice of how to sort and we will also perform pagination. We can limit the depth of the comments to about 2 levels and allow the user to load more if they choose. We can have the sort key be a path of commentIds (ex: id1/id2/thisCommentId) so that we can easily retrieve nested comments. This will be using the materialized paths method to handle deep nested loops without a full tree scan. If they choose to load more comments, we will continue to load about 4 levels of comments before the user needs to click load more.
   7. If there is a cache miss, we will set a lock key for the read cache so that a request will fetch from the database while other requests will wait for us to store the results into the cache.

Tech Choices and Trade Offs

1. Tech Choice
   1. DynamoDB, Redis, SQS, SNS, ELB
2. Trade Offs
   1. We have decided to use non-relational database over a relational database so that we can perform horizontal scaling. We are willing to sacrifice immediate consistency for better scalability and handling of high volumes of read/write requests.
   2. We have decided against using an S3 bucket to store comment contents because we would need to upload new files every time the user updates their comment. We will also significantly increase the latency from retrieving from an S3 bucket.