Loading...
DAU = 500M
write in a day = DAU * 1% = 5M
read in a day = DAU * 5 = 2500M
How large storage should we need?
size of origianl url = 100B
{
original url : 100B
short url : 8B
createdAt : 8B
createdBy : 8B
expiredDate : 8B
}
size of write in a day = write in a day * size = 5M * 132B = 660M/day
generateID : generate unique id which length < max length
getOriginalURL : get original url from shortURL
register user
login
getHistory : get history of mapping between original url and short url
index of mapping table
primary index : short_url
secondary index : createdBy
database choice
API Gateway
User service : receive register and login request and store in MySQL
id generator : generate unique id from nano time for now
mapping service : generate short url through id generator and store in redis, MySQL. And get mapping from Redis redirect to original url
cache layer : redirection service must have high performance. we introduce 2 way cache layer. one is inside of data center, one is outside. we can store most frequently accessed data to CDN. it would have small set because of high cost. and redis would have larger set of data. both of them have
cache
performance of redirection api is extremly important for this service. we employed 2 phase of cache. because request have locality nature of access, CDN is effective. at the closest location from users, we would have CDN storing short->long mapping data for most frequently requested urls. for example celebrity post the short url in their social network post, It would be stored in CDN and most of request would be handled by CDN even not reaching API gateway. it is quite beneficial for scalability and fault tolerance perspective. it has limited storage, small part of data would be stored.
in the data center, we employ a caching node(e.g. redis). we can install multiple redis node, we can store a larger set of mapping. it is still faster than accessing to database. it has performance and scalabiltiy gain
Both CDN and redis would use least recently used eviction policy to ensure currently popular mapping stay in cache
creating short url can be 2 ways.
Both have pros and cons. pros of using hash is we don't need to generate random value but cons is they might collide. And our short url(8B) is more shorter than result of hash(20B). they might collide often.
on the other side, pros of generating random is we can avoid collision by regenerate random. but the cons of it is we cannot support custom value for short url.
However in this service, generating random is more suitable. we can support mapping different short url for same original url in case user want to analyze for that.
All of component like load balancer, api server, database can fail.
Mapping service
If mapping service fail, time sensitive request would be affected. (e.g. redirect url) As the service is stateless, we can put multiple nodes and using kubernetes user can access only to healthy node.
cache fail
If redis fail, mapping service have to access database. Load of db would increase and query would be slower. service would retry after time out. db load and latency increase more and more resulting in db crash.
To mitigate this, we can put 2 read replica for 1 lead node of redis. write would be handled in load node and propagate to read replica through write log. And If load node failed, one of read node would be lead node.
supporting custom url