Loading...
High Availability : user should be able to use our system anytime.
Consistency : eventual consistency should be fine.
Latency: low latency
Reliability : our system should be generally reliable
DAU : 1M 5 comments per user daily
Storage : 5M * 0.2KB per comments - > 1K * 1M = 1GB daily 400GB per year
post comments per second : 5M/100k = 50 comments/ second
Peak QPS 500/ second
POST /v1/comment
JWT header with userID and authorization
parameter : comment
return : 400s 200s
GET /v1/comments/videos/id
JWT header with userID and authorization
parameter: video ID, timestamp
return real-time comments for this specific video
POST v1/comments/stop/video/id
JWT header with userID and authorization
parameter: video ID
stop loading comments for specific video
comment:
-commentId
-userId
-content
-videoId
video:
-videoId
-length
-uploader
...
user-
-userId
-videoviewing
-lastCommentTime
post comment data flow:
client sent a post comment request
request goes to API gateway (authentication, rate limiting, load balancing),
then request goes to comments service and service store the comment in database
get comment data flow:
client sent get request to get all the comments for a video.
request goes to API gateway (authentication, rate limiting, load balancing)
then request goes to request goes to comments service and start responding realtime comments.
close comments flow:
client sent get request to get all the comments for a video.
request goes to API gateway (authentication, rate limiting, load balancing)
then request goes to request goes to comments service and comments service stop sending real-time comments
post comment data flow:
client sent a post comment request
request goes to API gateway (authentication, rate limiting, load balancing),
then request goes to Comments Service and service store the comment in database
get comment data flow:
client sent get request to get all the comments for a video.
request goes to API gateway (authentication, rate limiting, load balancing)
then request goes to request goes to Comments Service and start responding realtime comments.
close comments flow:
client sent get request to get all the comments for a video.
request goes to API gateway (authentication, rate limiting, load balancing)
then request goes to request goes to Comments Service and comments service stop sending real-time comments
How to get comments real-time:
subscription redis cache contains sets of users who in the same chatroom or in the same chat session of a video.when Comments Service receive a comments, it should be cached to redis, then check subscription cache and find the users who subscribed the video chat, then send the comments to all the users.
How to get older comments:
when load old comment, it request the comments from specific video ID and from the lastCommentsTime from user table, it fetches the 100 comments to that timeStamp then update user table lastCommentTime.
How to stop real-time comments:
when user exit a video, cancel request sent to Comments Service with a video ID and user Id, then subscription redis cache clear remove the specific user from cache
Cache:
when comments comes into Comments Service , it first write to cache for fast write. then it period flush the new comments to database.
how to handle cache fail ?
use white-ahead-log and period take snapshot of cache to handle cache failure.
cache sharding:
since 1GB daily should handle most case without shading, but in case popular videos and tons of comments on it. we can shard by video id so all the comments goes to same shading to faster retrieval.
write cache :
we separate write cache from write cache for improve latency and overall performance.
eventual consistency should be fine.
hot key issue:
We can store the same data in multiple keys and randomize the requests so they are spread across the cluster.
Servers, cache and database has redundency to handle failure scenarios.
Might be good to add monitoring system to monitor system healthy.