Loading...
If we establish a 5 year TTL, then for 30 million URLs a month over 5 years, that means 1.8B records. If the average URL is 70 characters, our service shortens it to 15, and we store metadata (let's assume 20 characters) then each record may be at least 100-200 bytes. This means about 0.3 TiB per year assuming average URLs, or 3 TiB per year if we allow 700-character URLs. But because URLs tend to run longer (otherwise, why use the service), perhaps let's round up to 5 TiB per year, which is ok.
CreateUrl(url) returns Result
GetUrl(shortened_url) returns Result
CreateUrl would take a long URL, and return a Result object containing the status code/error, etc. -- if successful it would also return the shortened URL.
GetUrl takes the shortened URL in CreateUrl and returns the originating URL within a Result object, but will also return error/status code if the URL is malformed or does not exists.
The URLs should be separate from each other, and the date of the URLs doesn't really matter. So, we don't really care that much about ordering. MongoDB should work well for this.
As more and more people use the service, we likely are going to want to shard our data. Such a sharding key could be the first character of the shortened URL, as our hashing scheme should be random enough to scatter our records.
This would be a RESTful API that takes in originating URLs and turns them into shortened URLs, but also takes shortened URLs and returns the original URLs.
Upon shortening a URL, the algorithm used would be base62. Base64 is a more common scheme, but it includes non-alphanumeric characters, and we can assume that for sporting events/posters, people will be typing the URLs manually. We can go lower than 62 for ambiguity purposes (e.g., capital I and lowercase l may be swapped out; as well, 0 and o) as we likely will have users writing these URLs manually or using fonts with poor character distinguishability.
In addition to that we would need an actual hashing algorithm. We don't necessarily care about the URL being derivable from its shortname, however we should follow principle of least information, and hash the URL first (either with md5 or sha), and then convert that over to the base62 representation. this is good because the database record would store the original url anyway, so we don't need to care about collisions. it also means our urls will be consistently-lengthed. i think the proper choice would be sha1 as it has low collision risk.
Upon CreateUrl, the hashing+base62 conversion is done, then a record is stored in the database, keyed by the key mentioned earlier.
Upon GetUrl, the key is used to find a record if it exists, returning the original URL if it exists.
Additionally, we can assume that very popular URLs exist that wouldn't necessarily need a cold database query: we can cache the requests and use a load balancer to make sure these requests don't storm the service.
In addition to that we would need an actual hashing algorithm. We don't necessarily care about the URL being derivable from its shortname, however we should follow principle of least information, and hash the URL first (either with md5 or sha), and then convert that over to the base62 representation. this is good because the database record would store the original url anyway, so we don't need to care about collisions. it also means our urls will be consistently-lengthed. i think the proper choice would be sha1 as it has low collision risk.
Upon CreateUrl, the hashing+base62 conversion is done, then a record is stored in the database, keyed by the key mentioned earlier.
Upon GetUrl, the key is used to find a record if it exists, returning the original URL if it exists.
Additionally, we can assume that very popular URLs exist that wouldn't necessarily need a cold database query: we can cache the requests and use a load balancer to make sure these requests don't storm the service.
Like stated earlier, I prefer sha1 over md5 because of its collision resistance.
If sha1 were to be broken and an actor managed to find collision/preimage attacks, then that could be an issue.
If the hashing algorithm fails, then another one would need to be made.