System requirements

Functional:

List functional requirements for the system (Ask the chat bot for hints if stuck.)...

1. Users are able to list / view comments on any thread. Nested threads will require clicking on the thread before viewing.
2. Users are able to add / edit their own comments.
3. Users are able to post anonymously.
4. Users are able to like a comment.

Non-Functional:

List non-functional requirements for the system...

1. Reply and tagging notifications are delivered in under 1 s.
2. Number of concurrent users supported - 1M
3. Latency to add / edit comments < 200mss.
4. Latency to view comments < 200 ms.
5. Threads are paginated to limit amount of data loaded at once.
6. Service needs to have 99.9% availability.
7. Limits on nesting to prevent abuse or system performance.
8. Parent and child thread integrity needs to be maintained and child threads would only appear under the parent comments.

Capacity estimation

Estimate the scale of the system you are going to design...

Number of estimated users - 100M

Read :write ratio - 1:100

Number of comments per day - 1M

Size per comment - 512 bytes.

Per day size - 512MB.

3 year projections - 512 * 1000 - 512 GB.

TPS during morning - 1M tps

TPS after work - 10 M TPS

API design

Define what APIs are expected from the system...

list_replies(post_id, page_size, marker) :

Makes a GET http call to /threads/{threadId}

Thread can either be associated with a post or a comment itself. Paginated API sorted by relevance.

post_comment(post_id, user, message, post_anonymously) :

Makes a Post http call to /threads/{parentPostId}

For posting replies on any thread including setting to post anonymously.

post_update(post_id, user, engagementType) :

Makes a Post http call to /threads/{postId}

Users can like the post or any comment. This api can also be used to update previously posted comment.

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...

We'll track data in 2 separate tables.

1. Thread - Each post is essentially a thread as well. Table will have primary key as postId which can auto generated UUID or incremental sequence number. Additionally this table will have an index on parentPostId to search child post for a thread.
2. UserLikes - Once a user reacts to a thread their reaction will be recorded in the table where postId will be the primary key and userId will be the sort Id since single post can have multiple reactions and use case is to limit reactions to a post per user. No index are needed for this table.

Depth field in Thread table is going to help us limit nesting of comments.

High-level design

Design consists of 6 components

1. API Design - Centralized global endpoint that can also serve as load balancer, rate limiter or authenticate users.
2. TrendingPostsService - Idea is to host certain popular posts in cache, so DB updates are processed to find trending posts / comments that can be served from cache.
3. CommentsService - This is the main backend for posting comments/ replies or viewing the comments.
4. Notification Service - This service maintains a web socket connection with active users through different devices and pushes updates from their network to users in real time.
5. Redis Cache - Cache will track most popular engagements in real time, this view will be few seconds outdated but provides much improved experience which is acceptable for a social media platform.
6. DB - We will be using a NoSQL database like Cassandra or DynamoDB to store our comments which can be scaled horizontally. Data will be partitioned by post id which will provide the most uniform distribution of data avoiding any hot partitions due to few users posting more comments than others.

Request flows

Overall there are 4 flows.

1. User makes a request to view comments for a thread / post. Request is redirected through API gateway to CommentsService which first looks up data in cache and if not found reads data from DB. Paginated response would be returned to the user along with a token to request more data.
2. User posts a reply to a comment. This will also be handled by CommentsService and a new entry will be created along with it's depth from the original post and a reference to parent post as well as root post.
3. Notifications - Each post reply/ reaction will be processed asynchronously to send out a notification to the entire network of the user / followers.
4. User Engagement - User will only be allowed to engage at most once per post.

Detailed component design

Real time notifications

There are two ways to send notifications, either we can have all clients poll for new updates frequently which can resource consuming especially on mobile devices since our users can be on different platforms. Instead we choose to implement a 2 way communication where users/ devices register to receive notification and then updates are delivered asynchronously efficiently.

Supporting nesting of comments

Hierarchical structure of comments is maintained by storing both parent post reference and the root post reference along with the depth of the current post in relation to the root post. Comments will be stored/retrieved in the order seen by the comments service. Customers only see comments from one level lower along with metadata to see if replies exists on any of the comments giving users a chance to view them as well.

Partitioning / Sharding

Data in both DB and cache will be partitioned by posts Id which allows horizontal scaling of capacity since all relevant data will be stored within single node.

Trade offs/Tech choices

Database

We're using NoSQL data base like Cassandra / DynamoDB here to support horizontal growth of data.

Read replicas

Each shard will be replicated to support higher availability.

Real time notifications

Using WebSockets to maintain persistent 2 way communication instead of long polling.

Pagination

Avoids loading too much data and also avoids overloading systems at once.

Failure scenarios/bottlenecks

Future improvements

1. Implementing