Loading...
There will primarily be two main API's
POST: /api/v1/shorten
Given a Long URL, returns the short URL with HTTP 201
We will use a hash function to hash the Long URL and save it as Short URL in the DB
pub fn(long_url: url) ->url {
hash_fn(long_url)
}
GET: /api/v1/long_url
Given a Short URL, returns the short URL with HTTP 301
pub fn(short_url: url) ->url {
get_from_database(short_url)
}
We will use RDBMS as our storage is under 5TB i.e. not significantly huge. Also, we have structured data.
We create a URL Table with id as PRIMARY KEY and indexed on both Short URL and Long URL.
Ens customer's requests will reach to our Load balancers which will go Geo Routing i.e. routing requests to Users closest origin/web-server, which will then serve users request from the Database (Given the request type and status as explained in API Design)
There will primarily be two main API's
POST: /api/v1/shorten
With Post we check in the database if the URL exists we return the Shorten URL else we create a new entry in the database and return the Shorten URL
GET: /api/v1/long_url
Since, this is Redirect, we can use HTTP 301 which will indicate that browser caches the response. if cache hits browser redirects to long url else browser checks in the DB and returns the shorten url , while updating the browser cache. This helps offload get requests from Server and latency remains low. downside we cannot do analytics directly and if needed has to be handled separately. May be then, we return HTTP 302 or we add Auth to our APIs and track analytics with auth endpoint. Given the current scope, I will keep HTTP 301 as it also saves us some storage which otherwise would needed a CDN and an extra cost.
We should take about, How do we keep the URL as short and how short could be the URLs?
1st lets see, what would be the max length of our shortened URL
we know we have alphanumeric chars i.e. 0-9a-bA-Z = 10 + 26 + 26 = 62
we need n length such that 62^n < 36.5 billion.
So,
62 ^ 1 = 62
62 ^ 2 =3,844
62 ^ 3 =238,328
62 ^ 4 =14,776,336
62 ^ 5 =916,132,832
62 ^ 6 =5.68×10¹⁰ which is way over what we need. So, we can safely mark that, our URLs will be of max size 6.
We can use SHA-1, MD5 type hash functions or we could use base62. I will choose base62 as it simplifies our design, because with SHA-1 and MD5 we will get a long url and thus we would be able to take say first 5 chars and then, we will need some type of UUID to avoid collisions (again that has to be distributed, which is another big ask). base62 will handle this easily for instance any 5 digit number we can see we will max get a 3 digit URL.
we can also talk about traffic handling;
we are using load balancers for traffic, we will use Geo Routing algorithm. This will ensure our latency remains low. We will also use reverse proxy to save the identity of our web servers
We have chose bas62 for our hashing. We have made a tradeoff as the URL length will not be same. For instance a url of length 5 will have size 3 hash and length 7 will have size 4 hash. As you could see, our URL length could be predicated. however, we make this choice carefully as with this our systems remains simple, we do not have to rely on another distributed system.
we have made a choice that, we will write through Database and every time we check if the URL exists, that could load our system. So, we will use a technique called Bloom Filter which runs on a set and is very fast. however, it could give us false positives, which makes our system eventually consistent. we could handle this with counter bloom filter however, that will increase memory consumption by 4 times.
we have also chose for GET /api/v1/long_url to return HTTP 301 against HTTP 302. This will sure make our system fast and help our servers offload, save on caching storage costs. However, we can't do analytics with this. Should we need to we could address that, with AUTH endpoints which may call another service that store users info, we can also caller user-agent info or cookies to drive analytics better