Requirements

Functional Requirements:

List the key functional requirements for the system (Ask the AI for hints if stuck)...

1. Users can post comments.
2. Users can reply to comments.
3. Users receive a notification when someone replies to a thread they are part of.
4. Users should be able to edit/delete there own comments. Once a comment is deleted, there reply should also be deleted.
5. Users can upvote or downvote a comment.

Non-Functional Requirements:

List the key non-functional requirements (performance, scalability, reliability, etc.)...

1. The system should capture real-time updates.
2. System should be scalable with low latency

Capacity Estimation

Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...

1. Read/write ratio - on an average 50/1.
2. Expect 1000 comments per min, 60000/hr, daily 1.5M
3. Peak traffic ~ 2 times average.

API Design

Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...

Post a comment/reply to a comment -

POST /comments

body {userId, commentId(optional)}

Edit a comment -

POST /comments/{commentId}

body {userId}

Delete a comment -

DELETE /comment/{commentId}

POST /comments/{commentId}/upvote

body {userId}

POST /comments/{commentId}/downvote

body {userId}

Notify about update

POST /notifications/{commentId}

Load a comment and there replies

GET /comments/{commentId}?limit=10&page=1&depth=3

High-Level Design

Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.

1. Load Balancer - Balances load and implements rate limiter
2. API Gateway - Implements web socket gateway and handles OAuth/JWT
3. Web socket gateway - This maintains real time channel for fast communication and handle real time updates
4. Kafka - This has multiple topics which handles broadcasting and asynchronous communications.
5. Comments Service - Handles comments CRUD operation.
6. Elasticsearch - Used by user to get a comment with different filters.
7. Redis Cache - Notification service uses cache to avoid directly hitting DB.
8. User Service - Handles user related crud operations.

Database Design

Define the data model. Identify the main entities, their attributes, and relationships. Consider the choice of database type (SQL vs NoSQL) and justify your decision based on access patterns...

DynamoDB (NoSQL) will be required as there is no strict ACID requirement and there will be very high load which will be handled by DynamoDB.

Comments

{

PK: POSTS#postId

SK: COMMENTS#CommentId

parentId:

author:

createdAt:

updatedAt:

upvote:

downvote:

depth:

}

Users

{

PK: User#userId

SK: metadata

lastActiveMetadata:

}

Top level comment fetching - GSI Solution: Create a sparse index named GSI_TopLevelComments.GSI PK: postId (String)GSI SK: createdAt (String / ISO 8601 Timestamp)

Fetch all reply to a comment - GSI Solution: Create an index named GSI_Replies.GSI PK: parentId (String)GSI SK: createdAt (String)

Detailed Component Design

Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.

Comment will store all its ancester in the sortkey e.g. COMMENTS#123#234#908 to avoid N+1 queries.

Once there is a comment added, all the users that are part of the comment ancestry will be sent a notification. This will be taken care with the help of WebSocket.

We will create a comment hash to ensure there are no idempotent comment submission.

For deeply nested retrieval, the default depth will be kept to 5. If a user wants to expand, they can do so and for every api call, 10 more comments will be loaded. Just like how we handle pagination, user can go to the last thread if required.

For Notification fan-out across instances, Redis Pub/Sub is the most efficient choice because it is entirely stateless and operates with sub-millisecond, in-memory latency.