Requirements

Functional Requirements:

• Create and share paste
  • the system should be create paste for user and share that paste via a unique url
• Expiration of paste
  • allow the users to set the expiration of paste, which, after exceeding assigned value the paste will be deleted
• Unique URL/ID generation
  • the user should be able to robustly generate unique id for each url coupling with collision prevention mechanism
• Paste Retrieval
  • users should be able to retrieval paste using unique url
• User anonymity
  • Allow users to create paste without needing to create an account to preserve simplicity.
• Update Paste
  • users should be able to update an existing paste
• Delete Paste
  • users should be able to delete an existing paste (immediate delete)

Non-Functional Requirements:

• Low response time
  • < 100 ms for saving text
  • < 100 ms for retrieve text
• Durability
  • the data must be retained until it expires
• High availability
  • 99.99 % uptime
• Scalable
  • able to handle traffic spike
• Security
  • data rest: user text is encrypted with standard algorithm such as RSA
  • data transit: implement TLS for the website
• Consistent of user experience
  • the paste must be available to user immediately after its creation.

API Design

External API

1.Create Paste

Method: POST

Endpoint: /{version}/paste

Request Body:

paste_text: string (required) : text included in paste

Response:

Success:

status: ENUM: SUCCESS

status_code: int: 200

paste_id: id: id of paste

Error:

status: ENUM: ERROR

status_code: int: 4xx | 5xx

2.Get Paste

Method: GET

Endpoint: /{version}/paste/{paste_id}

Path Variable:

paste_id: {id}: id of the paste

Response:

Success:

status: ENUM: SUCCESS

status_code: int: 200

Error:

status: ENUM: ERROR

status_code: int: 4xx | 5xx

3.Update Paste

Method: PATCH

Endpoint: /{version}/paste/{paste_id}

Path Variable

paste_id: id: id of the paste

Request Body:

new_text: string: new text

Response:

Success:

status: ENUM: SUCCESS

status_code: int: 200

Error:

status: ENUM: ERROR

status_code: int: 4xx | 5xx

4.Delete Paste

Method: DELETE

Endpoint: /{version}/paste/{paste_id}

Path Variable:

paste_id: id: id of the paste

Response:

Success:

status: ENUM: SUCCESS

status_code: int: 200

Error:

status: ENUM: ERROR

status_code: int: 4xx | 5xx

High-Level Design

Service Component:

• CDN: act as cache that is in closest proximity to user.
• API Gateway: responsible for authentication and traffic rerouting to the correct micro service
• Rate Limiter: part of api gateway, limit the number of request that can be sent to the server to prevent DoS attack.
• Load balancer: responsible for distribute load for each specific micro service
• Create Paste Service: micro service responsible for creating paste
• Retrieve Paste Service: micro service responsible for retrieving paste
• Delete Paste Service: micro service responsible for deleting paste
• Update Paste Service: micro service responsible for updating existing paste
• Unique Id Generation service: micro service responsible for creating unique id of each paste using secure random algorithm to generate alphanumeric string with length = 8
• Cache Layer: to quickly retrieve frequently used data, instead of searching in the database which is quite slow.
• Database Cluster: cluster of database, act as source of truth for the system.

Diagram

Detailed Component Design

Gen UID Service:

• secure random
  • secure random to generate alpha numeric string whose length equal to 8 (1/62^8 probability for collision). In case of collision happen we will random string again.
• KSG algorithm

Database:

• database choice:
  • primary function of paste-bin is to map the unique id of paste to the text
  • key-value database:
    • we can directly used key-value database like map data structure to map uid (key) with text (value)
  • mysql database:
    • we can have the uid as the primary queue in the table while the text can be just a column in the table
  • in my humble opinion, I think mysql database is a superior option. Despite mapping uid with text is a primary objective. We need to go further than that. As we also need to keep track of the creation timestamp, expiration timestamp, title, isOneTimeRead, etc.
  • In this case, my sql is more scalable than just a key-value database.
• database instance:
  • only one single database instance to act as source of truth for the system
  • simple to implement
• database cluster:
  • multiple instance working together to improve availability of the data for the system
  • possible out of sync data
    • solution:
      • raft algorithm
      • assign the most updated database as a leader
      • the rest are follower and will replicate data from leader via log
      • if the leader become unhealthy elect one of the follower to become leader
• in this case we will go with database cluster as it is more scalable and provide more availability and its drawback from its distributed system can be mitigated by raft's algorithm

Paste expiration:

As we allow the user to set expiration condition for every paste we need efficient way to invalidate a balk of paste. For time condition we just need to map expiration_date with the id. If there is request after expiration timestamp we can easily know that this paste is invalid and delete it. In addition we can have the cron job that will periodically check if at the current time are there any paste that is expired if so we can delete them all for reducing storage cost.

Cache:

• we can use cache write through to keep the data in cache sync with the data base. Also we need test and set Time To Live for data in cache so that after the reasonable time the item will be clear from cache saving space. In case the data is deleted in the database we also need to set up trigger to make sure that the data is also deleted from cache.
• Algorithm
  • LRU: least recently use cache is one of the more popular cache algorithm thanks to its simplicity. Basically, it just have predefine size and if cache surpasses this limit it will remove the least recently used item from cache to make room for new item. This algorithm is one of built in algorithm that Redis has, however, the usage pattern isn't entirely suitable for this algorithm as it doesn't account for the how frequent it is used.
  • LFU: least frequently use cache. As the name suggest it is like an extended version of LRU as it also take the number of item usage into account. So, it is more suitable for our usecase.

Scalability:

We will deploy stateless service on container orchestration service such as AWS ECS. In this case we can scale up/down our container on a whim. We can also set up condition for scaling such as when the request amount of certain service exceed our predefine value. They will add more instance to serve the upcoming load. Also we can adopt fault tolerant architecture by using spot instance to save the cost of infrastructure.