Loading...
should the number of requests per second = 10.000 r/s
for saving the urls, let's say we're using rdbms database, if every url is to be saved for a long time (no expiration date), let's say 100 years,
1 url size ~ 300kb
100 x 365 x 86400 x 300 = 946080000000 kb ~ 946 tb
should we use redis for faster performance -> using 2:8 rule -> we're using redis 20% of our capacity
20% from 946 tb = 189 tb
shortening :
"https://shorturl.com/api/{user_id}/create"
user_id -> for authentication (if guests then default from UI)
long_url -> long_url that is to be shortened
custom_url -> if user want to shorten url with customization
expiration -> optionable, if user want to set expiration date, if null then don't set expiration
redirection :
"https://shorturl.com/api/{user_id}/get"
API should check first inside redis for faster retrieval, if doesn't exist inside cache, then can check inside the postgresql
should return 301 redirect
should use SQL over noSQL for saving the short url and long url since we want to prioritize the ACID transaction. Let's say there is 10.000 requests/second, we want to make sure there aren't 2 or more short urls with different long urls since it should be unique. Then, for this, we want the ACID side of relational database. PostgreSQL or MySQL can be used. For caching, we want to use NoSQL database like memcached or redis. Additionally, if we have several servers around the world, we want to make sure that we have different ranges of counter for different servers, that's why we use a pre-created short url from short url creation service.
table user (
primary key id int,
username varchar,
email varchar,
password varchar,
expiration_date,
)
table url(
primary key id int,
long_url varchar,
short_url varchar,
user_id foreignkey user
)
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
URL shortened flow :
Client requests to any of the server (load balancer provided, may use algorithm to balance the requests between different servers), server then would call shortening service to create short url from the request. First, validate the long_url, if it is not valid, return error. Check if customize_url exists, if there is customize_url inside the request -> check first from relational database, if it has already existed inside database, return error, if it doesn't exist, then directly create the short_url and save inside redis with expiration date customized from user(if not customized, then don't set expiration date).
Redirection flow :
Client hit the short url. Then our service would be hit, it then call the redirect service, service would check to redis first, for faster retrieval, if it doesn't exist inside redis, then it would check inside database. If it exists, then return 301 to redirect the user. If it doesn't exist, then return error. Then every time there is an url exists, we want to fire event to kafka stream, to provide metrics like which geography that our users are reside, which clients are using our services the most then we want to create CDN for faster performance.
Cleaning up flow :
We should set a scheduler every, let's say every 1 month to delete short url that is already expired to minimize the size of the database we're using.
There are 2 considered algorithms for shortening url. 1 base-64 shortening, and unique counter algorithm. This time, i would use the unique counter algorithm, this would set different range for different server, let's say server 1 use range from 1-5000, server2 use 5001-1000, etc. When server is running out of range, then it would be set another range to make sure there won't be any duplicate short url.
Explain any trade offs you have made and why you made certain tech choices...