System requirements

Functional:

1. Convert long url to short url
2. Persist the mapping between short and long url
3. Based on short URL get longer URL and redirect user

Non-Functional:

1. Response latency should on or below- 200 ms
2. Millions of URLS
3. High concurrent user access
4. High availability
5. Scalable system - Horizontal scaling
6. low redirect latency

API design

1. createShortURL : Input as long url and out put as Long url
2. getLongURL : Input as short url and output as long url
3. redirect:input as short url and redirect long url

statistics: input as short url and output as statistics number like

Analytics endpoints

1. /analytic/{shorturl} : input as short url and output as analytics details like Geo information, devices accessed , most accessed times

High-Level Design

1. Entry Layer - WAF - for all OWASP prevention, ddos attacks , forward request to API Gateway
2. Entry Layer - API gateway - gateway route requests to load balancer
3. Entry Layer - Load balancer - route requests to specific service
4. Write Path - User would like to generate Short URL, user makes the short url service with the user credentials and url
5. Write Path - Validation - Checks the URL already available , check the character set for any not allowed characters like sql injections ,
6. Write Path - Short URL Gen service - Encode url to avoid special characters , uses the hash function CRC 32 to generate the hash , Check for hash collisions , if not persist the data
7. Read Path - When user clicks short url , requests come to waf and gateway to Redirect service ,
8. Read Path - Gets the data from cache or from database. First it checks the cache if data available in cache if not asks the database and persist it in cache
9. analytics - Each request creates events through message broker , and that data will be consumed by analytic service
10. No CDN - It doesnt require any CDN as its doesnt have any static or big urls
11. Storage layer - Stores the short url , long url and expiry dates..
12. Cache layer - as it is high read system caches most frequent reads with ttl

Detailed component design

• Hash-based ID algorithm such as base62 or random generation

1. hash based generation with base 62
2. CRC 32 generates 8 character hash
3. Base 62 of 7 creates 3.5 trillion unique keys
4. those are good to start
5. hash key generates 8 character long , so take first 7
6. Random generation using UUID - length of uuid is 128 bits
7. converts typical in to 22 characters of base 62
8. we dont need that many characters , we need to take first 7 charcters
9. might have collisions UUID and high chance of first hash 7 collisions
10. Better to generate from the given long url

• Collision detection

1. considering the first 7 charter might create collisions
2. Check data base for any collisions

handling collisions :

1. 1. Append user id with url and generate hash
   2. check again for Collision

• Distributed vs central generator

1. Distributed generator as it requires local

• Edge CDN rationale

1. not required its costly and of no help

• Data-center cache (Redis)

Cache the short url as key and long url as value

helps for faster look ups with defined ttl

• Eviction/TTL policy

• evict if any changes on short url
• TTL as 24 hours

• Per-IP or per-token policy

1. Rate limit based on the ip address

• Burst handling/back-off strategy

1. Increase the node//pods based on the incoming requests

Database design

1. generate short url from uuid from long url string
2. To scale the database need to higher capacity
3. As the data is create only , we cna use nosql databases like cassandra
4. Database design like : long url, short url ,created time stamp , expiry time stamp , user id
5. User details : User id, user name , create time stamp, update time stamp
6. Use normalization to refer to data about meta data like user , domain
7. Partitioning Key as hash of short url , as hash of url are based on UUID , short url will be evenly distributed across shards, looks up also distribute across the shards
8. Partitioning based on hash using crc 32
9. Index on Short URL column help for faster look ups
10. Eventual consistency is ok for this use case
11. Data Normalization - use Normalization Users and URL