Loading...
100 000000 /(24 * 3600 * 30) = 38 URL creates per sec
100 0000000 / (24 * 3600 * 30) = 380 redirect per sec
Read Write Ratio 10:1
So assume 38 * 10 URL creates per second in peak traffic
3800 redirect per sec in peak traffic
Each URL record: short key (6-7 chars) + long URL (~200 chars avg) + created_at + user_id ≈ 300-400 bytes
With 100M URLs/month, that's ~30-40 GB/month → ~360-480 GB/year
POST /v1/api/create
request:{
original_url
}
response:{
statusCode
message
id
shortern_url
}
GET /v1/api/{shortern_url}
response:{
statusCode: 301/302/XXX
message
redirect_url
}
We have API Gateway responsible for rate limiter, authorization, routing and middleware.
So the workflow would be client POST /v1/api/create to API servers, API servers has business logic which generate shortern Code by using base62.
We have couple choices for shortern code algorithem, md5, base64, base62,etc. md5 is an algorithm has flow which can be decoded. base64 is good but it has some unsafe characters such as %, base62 is only contains a-z and it's enough for 360-480 GB a year with 6-7 chars for each.
Once shortern url generated, save the shortern_URL and original_url mapping in URL database.
once user want to call one of shortern URL, GET /v1/api/{shortern_url} will send requests through API servers, so there is 2 ways to redirect url to third party servers which return status code 301/302
301 is permanent redirect for those hot key, which means our API servers need to fetch redirect url from redis clusters/url database, then hit third party servers in the first time, then browser will cached redirect url, it won't hit our api servers next time to redirect this website
302 is temporal redirect, which means we need to hit API servers to find redirect url every time
So Database would be simple, we only need URL table looks like
URL:{
id
shortern_url
original_url
created_at
updated_at
}
So since we considering this is read heavy system(redirect),and availability > consistence, NoSQL is a better choice because we don't need to pursue ACID properties. I would like to choose DynamDB or MongoDB since they both implemented or partial implemented by B tree.
So i would like to choose the redis clusters to deep dive, in order to pursue low latency and high availability, we need to use leader and replicas macheism for redis clusters, using consistent hash ring to calculate partition id of each replica node, to ensure shotern urls are distrubuted evenly and leader node know to find which url in which replica node. We can use Zookeeper or etcd in control plane to store the topplogy, all redis node can send heartbeat to control plane, if any node down, control plane compoent will recalculate consistent hash ring. In addition, we might have hot key read issues in our system, some popular website like amazon, facebook will visited more frequently. Thereforce, we need 2-3 replicas node particularly handling those hot websites, once these websites being visited, local cache can take partial traffics, the rest of those can hit to those replicas for hot keys.
I think DB clusters also leader and replicas machism, once cache miss, it will routing the request to hit DB to find redirect URL and write in redis