Loading...
Input: Long URL; Output: Short (customable) URL, with experiation date (customable);
Analytics: Keep track of clicks and users location;
API: Able to built in other applications.
Performance: 10.000 calls per second
Scalability: Horizontal - should be able to handle 100*10^6 shortened URLs
Reliability: 99.99% uptime
Security: storing urls and preventing malware links injection
Monitoring: Alerting system to quickly tackle issue
Data Backup: regular URL backups
Compliance: GDPR
User experience: Fast and seamless
Estimate the scale of the system you are going to design...
Input the URL to be shortened as a JSON file. Output: a dictionary in a JSON file containing the shortened URL, and creation and exprining data (all keys in the dictionary), click count and user location.
See Diagram to the right.
Also see Diagram to the right.
Also see Diagram to the right.
Also, since we have a GDPR compliance, we only send a request if the user agreed about its information beeing saved.
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
Having several servers we should be able to handle one of them failing quite well - this increases our uptime. Each server does the same and the load balancer distributes the incoming calls accordingly. We can scale horizontally quite well by adding more servers.
Depending on the time/traffic, the dataloader redistributes the traffic to one of the differently located web servers - this is done by looking at the time and having a predefined flag signaling high/low traffic in the servers region at that time. The 4 servers are in 4 equally spaced timezones.
The algorithm that gives back the shortened url takes many things into consideration: if its customised, check whether the alias is available, otherwise we ignore the alias (and give a warning to the user). Check if the URL already has shortened version, if yes, give that back and update the expiration date (reset), unless a custom expiration date has been given -> then we create a new entry in the SQL database.
For the predictive analysis we export the numer of clicks within a timedelta (say 1 second) as a dataframe to python and handle it as a time-series, on which we can perform different analysis. To reevaluate and improve the dataloading we customize metrics that reward a good redistribution based on the traffic forecast.
Uptime was important, that's why want to avoid single point of failure. Moreover, we opted to use SQL for logging since it's handy to then also use machine learning to detect where we can improve our workflow. On a low-level side we should also
Try to discuss as many failure scenarios/bottlenecks as possible.
We might have peak traffic hours, which is a bottleneck. One way to handle this is to let the data loader distribute traffic across servers in different geographical regions. This can also be used as a multi-region deployment that redirects the traffic somewhere else in case of a regional black out.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?
Took care of this using dataloader, malware detection. Using the logged data, we can extrapolate via analytics on which region we would like to further deploy webservers, i.e. horizontal scaling. We could also improve the traffic distributing by forecasting the traffic; this can be done via forecasting models/machine learning.