Requirements

Functional Requirements:

• A user can create a paste and receives a unique url to access the content.
• A user can set visibility of the paste to public, private, or unlisted.
• Other users with access can retrieve post content with the unique url.
• Users can set paste expiry and the content will be deleted after the expiration time has elapsed.

Non-Functional Requirements:

• The system has 99.9% availability.
• The system has low latency for paste creation and retrieval.
• Paste content is consistent and after pastes have been edited, the updated past content is readily available to users.
• System is resilient to unintended usage (spam and bot usage).

API Design

POST /paste

GET /paste/{paste_id}

High-Level Design

Components

• PasteRenderer - Responsible for rendering paste content to the user.
  • PasteRenderer fetches paste content for a unique url from the PasteStore. PasteRenderer is responsible for syntax formatting/highlighting of paste and returning a response to the API. The client can use response from the API with minimal translation.
  • PasteRenderer can be load-balanced/scaled to handle large number of requests. Frequent GET /paste/{paste_id} requests can be cached such that any transformations do not have to be repeatedly calculated. Cache can use a LRU cache eviction policy
• PasteStore - Responsible for creating/storing/retrieving pastes to a database.
  • Pastes can have multiple storage options based on paste type.
    • In-Memory database like Redis/Memcachedb for temporary pastes.
    • PostGreSQL/DynamoDB for main paste use-case.
    • S3 for larger paste objects and attachments.
  • Content can be cached for faster retrieval with a LRU eviction policy. Cache is distinct from PasteRenderer as same post content may be requested but undergo several transformations.
• TTLExpiry - Responsible for deleting expired pastes.
  • Background jobs that run on PasteStore data storage in order to remove paste content that has passed the expiration date.
  • Background jobs can be run hourly or daily depending on needs of the system. Alternatively, background jobs can be scheduled to run ad-hoc for sensitive paste content.

Detailed Component Design

• Abuse Prevention
  • Rate Limit based on IP or user id (10 pastes/min, 100 pastes/day)
  • CAPTCHA process during paste creation
  • Limit on paste size (1MB for text content, 5MB for images)
  • Obfuscation of personal information and data.
• PASTE ID generation
  • Last x digits of UUID -> Base-64 encoded.
  • PasteIDs can be pre-generated and assigned when POST /paste requests are made.
  • Paste ids are marked as deprecated for a period of time after paste content deletion. Paste ids may be recycled for re-use after time has passed.