Requirements

Functional Requirements:

• First, I will assume the number of users to ~1 million users. I will also assume that each user will on average paste ~1 per day, which will be in total 1M pastes per day in the system. I will also assume that each paste is text only and the content can be up to 10MB.
• Users can paste and upload text and store them for user selectable duration such as 1hour, 1 day, 1 week, never. i.e. TTL is selectable.
• Each paste is assigned a unique, unguessable shareable URL such as pastebin.com/aB4g5e
• Anyone with the URL can view the paste content.
• System automatically expires and deletes the pastes after the TTL, which has been set by the user.
• Assume that the service is read heavy. Lets assume read to write ratio as 5 to 1.

Non-Functional Requirements:

• Latency: The system will be read heavy, and we assume that the read to write ratio is 5 to 1. We will need to prioritize the read latency, < 200ms should feel instant. Lets assume that write latency is tolerable < 500ms.
• Scalability: Scalable to 10x projected load i.e. 10M pastes per day by horizontal scaling for no single point of failture.
• Consistency: Strangers with the shared link will read the pastes. No body is collaboatively editing. So eventual consistency is fine, which means that we can cache quite aggressively.
• Durability: Once the URL is generated, the paste must be safe. Data must be replicated, never held in one node memory.
• Security: Pastebin has no authenticaion walls so the URL is the access control. The keys attached to the URL should be unguessable random keys.

Capacity Estimation

Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...

API Design

The API will need read and write endpoints. For the write endpoints, we will need to find a way to make a create request, which we can use POST method. For the read paste, we can use the GET method. Upon deletion, we can use the DELETE method.

Create paste — POST /api/pastes Body: { "content": "...", "expiration": "1day"} Response: 201 Created + { "key": "aB3xY9", "url": "pastebin.com/aB3xY9" }

Read paste — GET/api/pastes/{key} Response: 200 with content, 404 if expired/not found

Delete paste — DELETE /api/pastes/{key} (requires owner token for anonymous users) Response: 204 No Content

High-Level Design

There will be two flows. For the write flow, the user will submit a text, system will generate a key, stores that into the database and the system will return url. Let that service be called the URL generator application. For the read, the user will be able to look up the key from the database and fetch the text. Let that service be called text fetcher application.

In order to horizontally scale and to ensure that there is no single point of failure, we will add a load balancer between the client and the API gateway. The load balancer spread incoming traffic to multiple servers. We have assumed that there will be around 1M pastes a day and that the paste is 10KB. This means that 10GB/day, within a year this will be 36 TB/year. Wwe can have a relational database e.g. PostgreSQL for metadata. The actual text content can be stored in the object storage. Object storage scales to petabytes at cents/GB. This enables independent scaling and for managing the costs independently.

Database Design

The database should be postgres.

Detailed Component Design

Component 1: Key Generation

Responsibility: Produce a unique, unguessable 7-char key for every new paste.

How it works: Random base62 (0-9A-Za-z) key, 7 chars → 62⁷ ≈ 3.5 trillion combinations. For ~1M pastes/day (~12 writes/sec), collision probability is negligible; on the rare collision, regenerate and retry.

Uniqueness (race-safe): On write, INSERT with a UNIQUE constraint on key. If the DB returns a duplicate-key error, the app generates a new key and retries. No scanning — the index makes it an instant lookup, and the constraint guarantees correctness even with concurrent creates.

Unguessable: 3.5T keyspace makes enumeration infeasible → satisfies the NFR that "the URL is the access control."

Scaling: Stateless — any app server generates keys independently; no shared store needed at this scale.

Trade-off: Random+retry is simplest and unguessable. Sequential base62 counter is collision-free but guessable (violates the NFR). Pre-generated key pool is fastest but adds a store — overkill at 12 writes/sec.

Component 2: Read / Cache Path

How it works: GET /pastes/{key} → check Redis first. Hit → return content (no DB/S3 touch). Miss → read metadata from Postgres → fetch content from S3 → populate Redis → return.

What gets cached: the content blob, keyed by paste key, with a TTL ≤ the paste's expiry. Hot pastes then serve in ~1ms from memory.

Cache-miss path: Postgres holds metadata; S3 holds content. Both are only hit on a cold read, so the read-heavy load (10:1) is absorbed by Redis.

Scaling: Add Redis nodes / shard by key hash; read replicas cover cache-miss reads.

Failure scenario — thundering herd: a viral paste's cache entry expires → thousands of requests hit the DB at once. Mitigate with a lock-on-miss (one request rebuilds the entry, others wait/brief-stale) or serve slightly-stale content while one worker refills.

Graceful degradation: if S3 is slow or the DB is down, cached pastes still serve — only cold reads and new writes fail.

Component 3: Cross-cutting Concerns

Idempotent creation: client sends an idempotency key; if a retry arrives with the same key, return the original paste URL instead of creating a duplicate. (Alternatively, dedupe on a content hash within a time window.)

Rate limiting: token bucket per IP (~10 creates/min) to stop abuse/spam — important since creation is anonymous.

Expiry / cleanup: a cleanup worker scans the expires_at index, deletes expired rows from Postgres and their content from S3, and evicts the cache entry.

Trade-offs to name aloud: content in S3 vs. inline in Postgres (independent scaling + cost); Redis eviction policy (LRU) since it's a cache, not the source of truth; eventual consistency is fine here — a just-created paste may 404 briefly on a lagging replica, which is acceptable for this product.