Loading...
Create short URL – input a long URL and receive a short URL in return
Redirect to original URL – when short URL is accessed it should redirect the users to the long URL
Track clicks – service should record analytics such as number of clicks on each short URL, timestamps of the clicks, and geographical location of the users
Custom alias – Optionally the users should be able to create a custom alias instead of randomly generating one
Expiration – expiration date for URLs
User management – login – create accounts manage their shortened URLs
List non-functional requirements for the system...
Scalability – handle an increasing amount of URLS and the traffic as the user base grows. Should be able to support high volume of URL shortening request and redirects
Availability – Highly Available, minimal downtime, up 99.9%
Performance – redirect response should be quick ideally under 100ms
Security – Service must ensure security of stored URL, - project against malicious links and spam
Data integrity – Ensure the mapping between the short URL and long URL remains consistent even during system failures
Usability – system should be user friendly for both end users and admins
Compliance – service has to comply with relevant regulations for data privacy GDPR
For our design exercise, we might plan for an initial capacity to handle around 10,000 to 50,000 users daily. This would provide a reasonable starting point, enabling considerations for scalability in the future.
shorten URL
10000 x 1.5/user = 15000 shorten requests/day
50000 x 1.5/user = 75000 shorten request/day
Daily redirects
10000 x 3URLs/users=30000
50000 x 3Urls/users= 150000
Lower bound request
Shorten = 15000
Redirects = 30000
Total lower bound = 45000 request/day
Upper bound request
Shorten = 75000
Redirects = 150000
Total upper bound = 225000 request/day
Estimate traffic
Shorten URL 200 bytes
redirect response = 100 bytes
Lower bound shorten traffic estimation = 15000 x 200 bytes = 3MB per day
Upper bound shorten traffic estimation = 75000 x 200 bytes = 15MB
Lower bound redirect traffic estimation = 30000 x 100 bytes = 3MB
Upper bound redirect traffic estimation = 150000 x 100 bytes = 15MB
Total data volume
Shorten URLS = 6MB
Redirects = 30 MB
Estimated Database storage required for URLs:
Average length of a URL is 100 bytes
short URL 10 bytes (after encoding)
Metadata 50 bytes
100 bytes + 10 bytes + 50 bytes = 160 bytes
Daily storage
Lower bound storage = 15000 x 160 bytes = 2.4MB
Upper bound storage = 75000 x 160 bytes = 12MB
Database storage overtime - assume storage for a year
lower bound = 2.4MB x 365 = 876 MB
upper bound = 12MB x 365 = 4.3GB
Define what APIs are expected from the system...
Endpoint
Request body
Response
Redirect to original URL
Track clicks
User Management
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
For Database DynamoDB would be my pick because it fits the requirements to scale automatic with traffic covering any spikes, offer low latency read and write operations under specified time requirements 100ms. We can use Key-value structure where short URL is the key and long URL is the value. We can use DynamoDB streams to capture changes to items which can be processed for analytics. User data can be stored with DynamoDB. Security is also covered for GDPR. For data integrity and consistency we can implement transactions in dynamoDB to handle the creation of short URLs and metadata.
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
My high level design is
Client connects to Load balancer layer connecting to Gateway API for requests which sends to backend server for applications which is connected to DynamoDB databases and also some caches to reduce latency.
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
The request would begin at the client and flow to the load balancer layer. Next the Load Balancer would this request to the API Gateway which would direct based on request type to the backend servers. The Backend servers would either hit the cache or database(s). The request would then go back up to the client.
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
The Load Balancer layer would be Stateful layer 7 active active horizontal scaling solution to ensure as the business grows we have redundancy, performance, and scalability. I would use layer 7 to ensure session persistence is possible. This fulfills our functional and nonfunctional requirements at this layer.
For the databases I would use DynamoDB based on the functional and nonfunctional requirements. This provides scale, performance, fully managed service, Key-value and document store solution, DynamoDB streams, Security features that fulfill GDPR, and multi region support.
For the caching component I Would use Memcached. Memcached is optimized for simple key-value storage and is extremely lightweight and fast, making it ideal for straightforward caching use cases.
Explain any trade offs you have made and why you made certain tech choices...
Load balancer layer - This choice comes with the tradeoff of complexity as this requirements more complex configuration to failover vs active/standby solution.
Database layer - This choice comes with the tradeoff of a database that has more complex queries like joins or aggregations.
Caching component - I choice comes with the tradeoff of limited data types and data persistence.
Future bottle necks
High read or write loads may cause slowdown on the primary database but that is where I've added Memcached as an option for future development.
Throttling or rate limiting failures maybe an issues. In this case we would implement rate limited at the load balancer and API Gateway. This would help with both rate limiting and DDoS attacks.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?
In the future I would implement CDNs to reduce latency getting closer to the customer.