Loading...
Shortening URL when given a long url
Redirection at given short URL
expired time
Unpredictability
High availability at peak hour
Scalability --> can be scale up or down based on the number of requests
Consistency
High performance , low latency
Read:write 100:1
200m new shortening request per month
50 bytes / reqeust
Monthly storage = 200m * 50 = 10GB
Daily Storage = 10GB / 30 = 330M / day
write QPS = 200m / 30* 10^5 = 67
read QPS = 6700
so it's read heavy
incoming traffic = 67 * 500 Bytes = 33.5KB/s
outcoming traffic = 6700 * 500 BYtes = 3.35MB/s
shortenURl(longURL, api_key)
redirect(shortURL, api_key)
response status code 301/302, 301 for the temporary redirection with the TTL, 302 for permeant redirection
api_key is to help limite the request for each user
Although the system looks like not need too much storage, but due to the read-heavy system, easy scale in the future and no relationship between the data, I will choose NoSQL database as our storage. Since it's not write heavy, MongoDB should be suitable for our design, it's using Primary-secondary to replicate the data, so fits our needs in availability.
In addition, we can also add a cache system to improve the performance.
we can have load balancers to evenly distribute the reqeust to web servers, web servers will talk with the rate limiter to check if it can accept the request. If approved, the web servers will forward the request to backend servers, which is consist of Shortening service, Redirection service. The services will handle the request and store the data like shortening URL to storage. Also, we have cache system to store the frequently accessed URL.
user send a reqeust will go through the load balancers, and then forward to web servers. Web servers will talk with rate limiter to verify the request. If invalided, will return error to user. If valid, will forward the request to app servers. The shortening and redirection reqeust will be sent to Shortening service and Redirection service respectively. The new shotening link will be stored in MongoDB. THe cache system will cache the URLs that are frequently accessed.
For the shortening service, how do we shorten the URL? The idea is that we can randomly assign a key, but it might have duplicate key issue. The another one is that we can convert the long URL to a unique ID which is base 10, and then convert it to base 64, we can do the remainder calculations based on the ID, and then use the mod to match the table and get the URL
How do we ensure the availability and performance? The MongoDB is using primary-secondary mode for the replication, all the secondary nodes can serve the read requests, in case the primary node is down, will select the secondary node as primary node. For the cache, we can use memcached due to simplicity and fast access. Although the redis is more powerful than Memcache, but it will increase the complexity of the system.
How do we setup the expiry time? We need to tweak our shortening api, we can increase a paramenter called expiry_time, default is 5 years. It will also be stored in the mongodb.
If there are more and more write requests, MongoDB might become a bottleneck because it's not write-heavy friendly.
If the cache system was down, all the read requests will directly access database, which increase latency.
Adding a monitoring system to check the health of our microservices.