System requirements

Functional:

• generate shortened unique urls from larger ones
• make sure when a short url is visited, it is redirected to the original one

Non-Functional:

• high availability for redirection
• low latency for redirection
• scalability to handle high traffic

Capacity estimation

• Determine how many urls the system can handle
• Determine storage needs
• Determine request frequency

API design

• Create short URLs from long ones
• Retrieve long URLs from short ones

Database design

• ID
• UserID (creator, FK)
• CreatedTime
• ShortURL (indexed)
• LongURL (indexed)

High-level design

Client sends request to API Gateway

API gateway routes requests to split of long->short, short->long microservices.

Long->short microservice actually creates the records and inserts them in DB. Returns the short url to client.

short->long microservice will have heavily cached urls in redis to prevent unnecessary DB access. Could also include a CDN if request volume is high enough or audience is very distributed. If url not in cache, pull from DB, cache result and return to client.

Request flows

Client browser sends request with long_url to shorten

• LB TLS terminates traffic, sends to long->short MS
• L->S MS calculates new short url, commits to DB, returns result to LB
• LB returns new shortened URL to user

Client browser requests short url

• Request hits LB, which has a local cache it can check against and return redirect if found.
• If no cache, send request to short->long MS
• S->L MS checks redis, returns cached result if present
  • If not, checks DB, and then both caches and returns result
• LB caches result locally and returns redirect from StLms

Detailed component design

• Given we need very fast redirection, and cached urls don't require any sort of authentication check, we can have lightweight short->long urls KV stores at the boundary and could possibly implement a CDN.
• Cache busting - we would need to consider cache busting if we were to enable the users to edit a long URL upstream

Trade offs/Tech choices

In the beginning, I don't think there would be a need for a queueing service for getting long->short urls into the DB. However, if we have a bunch of traffic and are worried about taking down the DB, we could implement a queuing system for requests to protect overload.

This queuing system would require the client to poll the server for getting the returned shortened URL, so that would cause more traffic.

There is also the question of having a DB system with eventual consistency, allowing for much faster writes, vs complete consistency. If we go the complete consistency route, we can be sure that the short->long mapping will be instantly available. This will be useful if someone is quickly generating a short_url and then sharing it via QR or any form of social media.

For generating the URLs we could either go with a hashing algorithm, or random generator. Either way, we'd need to check against the DB for duplicates and regenerate if a collision is reached. We could benchmark performance to see what is faster on the hardware.

Failure scenarios/bottlenecks

We could have failures at the LB level. This would be handled gracefully as the DNS will realize when LBs are down and adjust accordingly.

We likewise will have horizontally scaled the two microservices, so that should fail gracefully.

If we have a cache go down, it could greatly increase momentary traffic on the DB, which could bog up and cause a slow down or ultimate failure. I think this is our biggest risk. Having replica DBs would allow us to quickly select and promote a new master DB quickly.

Future improvements

We would want to consider scalability concerns, like optimizing a CDN for local redirecting caching.