Loading...
User should be able to create a shorter link for a URL
User should be able to go to original link using the shorter link
Short URLs should be unique for each user's link
System should be highly available, users should be able to follow the short links with high uptime
Eventual consistency is fine, if the short link doesn't work for a small amount of time after creation. But once its up it needs to stay up
System should scale to Millions of requests per second for URL clicks
Estimate the scale of the system you are going to design...
POST /shortenedUrl -> response: ShortCode
sessionToken:
{
"url":
"expiryDate": # Default value used if not provided
}
GET /:
For the database here I would use a NoSQL database to store the key value pairs of a shortUrl and the actual url. Since the data is unstructured NoSQL can provide us better performance with respect to Scaling and performance. Since we don't require hard transactions, a Relational DB would have a lot of overhead.
The shorten service is at the heart of the system where it creates and retrieves shortened urls. A centralized unique key generator makes sure that every new URL is mapped to a new key to represent it. They key is then assigned to the original URL and stored in the DB from where it's queried to send back to the user.
We reply with a HTTP redirect to the original URL
For a POST request:
The client sends the url in the payload, we query the key generator for a unique key and insert the URL in the database and send a response to the user with a new "link" which is basically
For a GET request:
We query the database for the shortCode as a key in the DB and send a HTTP redirect to the stored URL as a response to the user. If the current date is past the expiry date, we return an error and mark the record for deletion
The NoSQL database will be sharded and values will be distributed using a consistent hashing technique to have a more uniform distribution of keys and to have redundancy in case any of the DB deployments goes down to make the system more fault tolerant.
The we should have multiple deployments of the shorten service behind a load balancer to make the system more resilient to failures. Load can also be more evenly distributed among the various deployments.
We also have a cache to store values for the most commonly hit shortened urls to prevent a database query. We can have caches local to each deployment to store values that are appropriate for each instance's usage and traffic patterns.
Using a NoSQL database provides us with an easily scalable and high performance system at the cost of consistency and transactions. Our data is relatively unstructured and so makes a good fit for this kind of DB and our SLAs don't have strict real-time requirements; reliability and availability is more important for us and in both cases a NoSQL database is a better fit out of the box. A SQL database can also provide those properties, but with additional overhead and complexity.
We use a centralized unique key generator to make sure that multiple deployments of the shortening service don't produce overlapping shortened urls to prevent collisions.
Whenever a URL is expired, we check "just in time" if its expired or not. This keeps our system simple but for URLs that are accessed very infrequently we might have a lot of dangling records that have expired. We can mitigate the impact on the size of the DB by having a job that deletes expired records periodically.
The key generator can go down and cause a blocker for any new urls being shortened. We can deploy a backup generator that keeps in sync with the state of the primary deployment and can take over traffic in case the primary goes down. Or we can have multiiple deployements each responsible for a subset of the keyspace.
Add a data aggregation pipeline to gather metrics about usage of different urls.
Allow users to set custom url short codes