Requirements

Functional Requirements:

• First, I will assume the number of users to ~1 million users. I will also assume that each user will on average paste ~1 per day, which will be in total 1M pastes per day in the system. I will also assume that each paste is text only and the content can be up to 10MB.
• Users can paste and upload text and store them for user selectable duration such as 1hour, 1 day, 1 week, never. i.e. TTL is selectable.
• Each paste is assigned a unique, unguessable shareable URL such as pastebin.com/aB4g5e
• Anyone with the URL can view the paste content.
• System automatically expires and deletes the pastes after the TTL, which has been set by the user.
• Assume that the service is read heavy. Lets assume read to write ratio as 5 to 1.

Non-Functional Requirements:

• Latency: The system will be read heavy, and we assume that the read to write ratio is 5 to 1. We will need to prioritize the read latency, < 200ms should feel instant. Lets assume that write latency is tolerable < 500ms.
• Scalability: Scalable to 10x projected load i.e. 10M pastes per day by horizontal scaling for no single point of failture.
• Consistency: Strangers with the shared link will read the pastes. No body is collaboatively editing. So eventual consistency is fine, which means that we can cache quite aggressively.
• Durability: Once the URL is generated, the paste must be safe. Data must be replicated, never held in one node memory.
• Security: Pastebin has no authenticaion walls so the URL is the access control. The keys attached to the URL should be unguessable random keys.

Capacity Estimation

Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...

API Design

The API will need read and write endpoints. For the write endpoints, we will need to find a way to make a create request, which we can use POST method. For the read paste, we can use the GET method. Upon deletion, we can use the DELETE method.

Create paste — POST /api/pastes Body: { "content": "...", "expiration": "1day"} Response: 201 Created + { "key": "aB3xY9", "url": "pastebin.com/aB3xY9" }

Read paste — GET/api/pastes/{key} Response: 200 with content, 404 if expired/not found

Delete paste — DELETE /api/pastes/{key} (requires owner token for anonymous users) Response: 204 No Content

High-Level Design

There will be two flows. For the write flow, the user will submit a text, system will generate a key, stores that into the database and the system will return url. Let that service be called the URL generator application. For the read, the user will be able to look up the key from the database and fetch the text. Let that service be called text fetcher application.

In order to horizontally scale and to ensure that there is no single point of failure, we will add a load balancer between the client and the API gateway. The load balancer spread incoming traffic to multiple servers. We have assumed that there will be around 1M pastes a day and that the paste is 10KB. This means that 10GB/day, within a year this will be 36 TB/year. Wwe can have a relational database e.g. PostgreSQL for metadata. The actual text content can be stored in the object storage. Object storage scales to petabytes at cents/GB. This enables independent scaling and for managing the costs independently.

Database Design

The database should be postgres.

Detailed Component Design

1. Key generation: The key generation can be generated with base62 keys, which has 7 characters. The keys are generated randomly, so we can have a background worker check if the new key is actually unique by scanning the metadata database (PostgreSQL); this will avoid collision between keys.