Requirements

Functional Requirements:

• Create Comment
  • user should be able to comment/reply
• Retrieve Comment
  • user should be able to read comment
• Update Comment
  • user should be able to update the comment
• Delete Comment
  • user should be able to delete the comment
• Nested Comment Structure
  • the system should be able to display the comment structure with correct indent according to comment hierarchy
• Perseverance Of Chronological Order
  • the system should be correctly order of comment based on chronological order

Non-Functional Requirements:

• Low Latency
  • p99 latency < 250 ms
• Consistency
  • the comment data stay consistent across multiple users
• Highly Scalable
  • able to adjust its capability according to incoming load
• Durability
  • the data is retained until deleted
• Availability
  • up time > 99.99 %

Capacity Estimation

• Assumptions
  • Only text comments are allowed
  • About 10,000,000 users globally
  • Active users: 500,000 users
  • Each user make about 200 requests a days
• Load
  • There is 100 million requests/day
  • hence requests/second:
    • 100,000,000/(60*60*24)
    • 1157.40740741 ~ 1200 requests/s
• Storage
  • there is about 10 billion comments
  • in average each comment take 0.2 KB
  • hence 200 * 10,000,000,000 ~ 2 TB
  • add 2x error factor ~ 4 TB

API Design

External API

• Create comment
  • Method: POST
  • Endpoint: /{version}/comment
  • Path variables:
    • version: str: api version
  • Body:
    • parent_id: id: parent's comment id
    • content: id: content of the comment
  • Response
    • Success
      • status: ENUM: SUCCESS
      • status_code: int: 200
      • comment_id: id: comment's id
    • Service unavailable
      • status: ENUM: ERROR
      • status_code: int: 503
      • message: str: error's message
• Get Comment
  • Method: GET
  • Endpoint: /{version}/comments/{comment_id}
  • Path variables:
    • version: str: version of api
    • comment_id: id: id of the comment
  • Response
    • Success
      • status: ENUM: SUCCESS
      • status_code: int: 200
      • comment_id: id: comment's id
    • Service unavailable
      • status: ENUM: ERROR
      • status_code: int: 503
      • message: str: error's message
• Update Comment
  • Method: PATCH
  • Endpoint: /{version}/comments/{comment_id}
  • Path variables:
    • version: str: version of api
    • comment_id: id: comment's id
  • Body:
    • new_content: str: edited comment
  • Response:
    • Success
      • status: ENUM: SUCCESS
      • status_code: int: 200
      • comment_id: id: comment's id
    • Service unavailable
      • status: ENUM: ERROR
      • status_code: int: 503
      • message: str: error's message
• Delete Comment
  • Method: DELETE
  • endpoint: /{version}/comments/{comment_id}
  • Path Variables:
    • version: api's version
    • comment_id: comment's id
  • Response
    • Success
      • status: ENUM: SUCCESS
      • status_code: int: 200
      • comment_id: id: comment's id
    • Service unavailable
      • status: ENUM: ERROR
      • status_code: int: 503
      • message: str: error's message

High-Level Design

Detailed Component Design

Database Design

• nature of data
  • comment is only text data, however, its size is not limited
  • there is parent-child relationship between comments
• Choices
  • SQL database (e.g. postgres, mySQL)
    • Comment Table Schema
      • comment_id: id: primary key
      • s3_url: str: content's object storage url
      • parent_id: id: id of its parent comment if parent_id == -1 then it is root comment
  • noSQL database (e.g. neo4j, neptune)
    • we can use graph database to maintain parent-child relationship between each comment by
    • node as a comment
    • edge as a relation
      • source is parent
      • destination is child
• Decision
  • in our case, the design would benefit more from choosing graph database over traditional sql database as graph database is more suitable for representing relationship
  • neo4j also support replication using onlinebackup
    • raft algorithm for selecting leader-follower
  • neo4j also support sharding
  • hence we would want to pick neo4j as our database of choice

Cache

• cache is implemented to reduce overall load on database, by storing, comments of the more popular thread
• cache invalidation strategy
  • in this case, we may use purge cache invalidation.
  • every time we write data on the database we delete in on cache ensuring it will fetch it from the database for the next time

Nested Structure Algorithm

• In this case, we can use bfs algorithm for rendering the comment. When we fetch the thread we can fetch the 25 most popular root comment and cache their children. After that, if the user tap show replies for a specific comment we render its children and cache its grand children. Thus, the nested structure is established.