Loading...
List functional requirements for the system (Ask the chat bot for hints if stuck.)...
1) User should be able to Generate ShortUrl From LongUrl.
2) User should be able to redirect From ShortUrl to LongUrl
List non-functional requirements for the system...
1) The shortUrl generated Should be unique for each longUrl.
2) System should be available(availability > consistency)
3) The system shouldd be low latency i.e. creation of shortUrl and redirection to longUrl < 100ms
4) System should be scalable for read heavy scenarios such as to handle high TPS for popular redirects supporting 100 M DAU and 1 B Urls
5) System should be fault tolerant employing replication and failover mechanisms
Estimate the scale of the system you are going to design...
1 Billion URLs to be supported by our system
the row will be string the data like shortUrl(~6 bytes), longUrl(~100 bytes), creationTimeStmp(~8 bytes) and some metadata(~accounting to 200 bytes) so total ~400 bytes
1 Billion * 400 Bytes -> 400 GB space is needed
1M URLs created/day -> 1M/100000 -> 10 writes/sec
Assuming read heavy system read:write -> 1000:1
10000 reads/sec
during peak hours -> 100x -> 100 * 10000 -> 1M reads/sec
Define what APIs are expected from the system...
Write API(Create shortURl)
POST /v1/shorten :
RequestBody:
{
longUrl,
userID,
expirationTIme(optional)
}
ResponsBody:
{
shortUrl
}
Read Api(Redirect to longUrl)
GET /v1/redirect/{shortURL}:
ResponseBody:
{
longURL http REdirect 302, if want to cache then 301. Use 302 for plugging additonal features such as analytics, deletion/updation of URL etc
}
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
Entities
longURl
shortUrl
user
Table -
urls
shortUrl(PK) longUrl userId(FK) timestamp_creation timestamp_expiration
user
userId(PK) userName email password
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
Key components -
Client
Load Balancer
API GAteway
Shorten Service
Redirect Service
Redirect Redis Cache Server
Redis Global Counter
URL database
These are the key components in my design
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
POST /v1/shorten
request hits the load balancer -> routes the request to appropriate healthy API Gateway -> routest he request to shorten service -> shorten service generates the unique shorten string with the help of redis global counter -> stores the entry in url database -> return 201 created response code to user with below response body -
{
shortenUrl
}
Get /v1/redirect/{shortenUrl}
request hits the load balancer -> routes the request to appropriate healthy API Gateway -> routest he request to redirect service -> checks the redirect redis cache if the shortenURL -> longUrl K_V pair is present redirects the user to the corresponding URL -> if not present the request hits URL database -> if shortenURL not found returns 404 code -> if found redirects with 302 code to longerURL
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
LoadBalancer -> Multiple instances of load balancers across different availability zones, using AWS elastic Load Balancer or Google Cloud Load Balancer because they can autoscale and also have a health check mechanism for the API GTEWAY before routing the request to them
API GATEWAY -> Multiple instances of api gateway across different availability zones, auto scaling policies in case of peak traffic
CDN -> Geographical based multiple CDN instances to serve static content and edge computing for redirection logic i.e. if shortURl -> longURl is present in CDN redirect 302 else redirect to redirect service
Horizontally scalable Shorten as well as Redirect Service.
Url database supporting database replication across multiple instances
Database backup in differnet availability zones
Redis Global Counter -> Intorducing counter batching to reduce network calls for each shorten URl call, enabling build-in-replication
Alogirthm to generate Shortenurl
each shortenService will read Redis GLobal Counter for counter value increment the value store it to redis global counter, encode with base62 encoding(0-9,A-Z,a-z), store the url entry row in url database
Counter batching -> each server will request n counter batches from redis global counter
redis global counter auto increments the value of counter by n and provide it to server
server uses these n values without making further calls to redis global counter until values are exhausted
Explain any trade offs you have made and why you made certain tech choices...
Try to discuss as many failure scenarios/bottlenecks as possible.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?