System requirements

Functional:

• Create a short link
• Get Original URL

Non-Functional:

• High-availability

Capacity estimation

• Assuming 200 requests per second to shorten URLs
• Assuming 20000 requests per seconds of fetch + redirect shortened URLs
• Assuming the TTL is 48 hours to fit a mail campaign for instance, it represent 200 * 3600 * 48 records so about 35Milions active records in the DB.

API design

• createShortURL : Short an URL, optional TTL, Authenticated.
• redirectShortURL, fetch original URL and redirect to it. Non authenticated.
• listShortURL, Authenticated, fetch all created ShortURL for the user that does the query.
• UpdateShortURL, Modify TTL.

Database design

a User table, and a URL table.

User table contains everything related to the User and the URL table contain the mapping between ShortURL and Original URL.

We should consider to shard database table to divide the load on indexes to search and insert.

High-level design

Load-balancers balance the load between APIs.

API then, first query the cache or fallback on the Database via a pgbouncer to optimise the use of the DB.

A garbage collector handle removing the data from the db once the TTL is passed.

Request flows

• Client wants to create a short URL
  • Post an authenticated request to the API containing the URL to shorten.
  • Load balancer redirect to the most available API
  • The API generate a random string of 8 to 9 characters.
  • The API send the request to Pgbouncer (r/w connection) as a transaction.
  • Postgrsql insert in a sharded table.
  • The API once the transacton is commited send back the identifier to the client.
• Client wants to fetch an URL from a short URL
  • Load balancer redirect to the most suitable API
  • Api query the database with an Read Only connection and return the result.

Detailed component design

The APIs are scalable horizontally.

We can add further replica nodes on the database to scale more on the reads.

Trade offs/Tech choices

I've decided to rely on the capacity of postgresql to scale

No archiving, so no history, Once deleted it returns nothing.

Failure scenarios/bottlenecks

• Collision in the name should re-trigger a new unique identifier computation from the API.
• if the R/W db fail, Patroni elect the other node as master.

Future improvements

We could imagine to have citus to scale further more the DB on the writes and shard the read traffic as well, but to stay highly available it will require to have a patroni + replica for each shards.