Loading...
List the key functional requirements for the system (Ask the AI for hints if stuck)...
List the key non-functional requirements (performance, scalability, reliability, etc.)...
Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...
POST /api/posts
PUT /api/posts
GET /api/posts
GET /api/posts?postID=
POST /api/upvote?postid|commentID=
DELETE /api/upvote?postid|commentID=
POST /api/comment
PUT /api/comment
GET /api/comment
GET /api/comment?commentID=
POST /api/subreddit
PUT /api/subreddit
GET /api/subreddit
GET /api/subreddit?subredditID=
POST /api/follow
DELETE /api/follow
CDN: It is used to host the frontend of the service. The response is fast because it is in edge locations. Most of the time it handles 80% of the reads since it can cache as well.
API Gateway: To route traffic based on the endpoint getting hit. Used for authentication and for rate limiting the requests.
Kafka: It is used as a high throughput queue to store the requests that are received. With this even if there is a down time it is easy to recover as kafka will have the missed requests. User will receive 200 OK
Redis:To cache the requests and take care of concurrent redundant requests and reduce load on the db and response time. and to store sorted sets so that delivery is fast.
Blob Storage: Great way to store the media that is uploaded into the platform we can use services like S3 for this.
Feed Generator: Generates feed by taking in the incoming data from the CreatePost service and is used by the Feed service. We will be using the Hybrid fan-out strategy where we will push to every follower's feed for normal users and pull at read time for celebrity.
Feed service: Uses the Feed Generator service generated data. Checks the redis first if not hit goes to Mongo and then makes an entry in redis so as to the next hit happens successfully at redis itself.
Mongo: Document DB used for this use case.
ElasticSearch will be used for indexing which is used by the search service.
Comment Service: This service handles the comment creation and everything related to it. Be it a comment on a post or a reply to a comment.
Follow Service: This will handle all the follow counts. this will be a couter which is sharded and occasionally flushes the aggregates.
We will be using different DBs
We will be storing the images, videos or any content in S3 and delivery is done through CDN
Comments: Mongo
ID
UserID
PostID/CommentID
PresignedURL
CreatedAt
UpdatedAt
Users: Postgres
Id
CreatedAt
UserName
PasswordHash
UpdatedAt
Subreddit: Postgres
Id
Name
CreatedAt
Admin
AdminMail
Followers
Followers: Postgres
Id
FolloweeId
FollowerID
Posts: Mongo
Id
UserID
Title
Content
SubredditID
PresignedURL
CreatedAt
UpdatedAt
CommentCount
Feed Generator: Generates the feed for the user. Generates feed by taking in the incoming data from the post service from the following of the user and is used by the Feed service. We will be using the Hybrid fan-out strategy where we will push to every follower's feed for normal users and pull at read time for celebrity.
We will be keeping the comments in the db with its id and if it has any reply its id will be present otherwise it will be empty. In this way a relationship is maintained.
If a action fails we even retry it for a set amount of times lets say 3. If it fails more than that set limit it will go to the DLQ. We will be retrying using exponential backoff with jitter to avoid multiple retries at the same time. This will prevent any load on the respective service.
We will be having replicas to adhere to high availability so that when ever a service fails it's replica takes over. We can go even further by having regional failovers so that if a region goes down we can switch to the other region.
We will be using sharded counters for upvotes and downvotes to avoid hot spots created due to viral posts. we will be spliting it into N shards and increment independently. We will flush aggregates to durable storage on a timer.
The counters will be having idempotency key as well to avoid double counting. If a operation fails in the middle we will be having an idempotency check to see if its already done if so we will just return that result we stored.