System requirements

Functional:

• Users should be able to input a long URL and receive shortened alias
• This shortened alias should redirect the user to the original long URL
• The shortened alias should be customizable by the user
• The alias can have an expiration date set by the user
• Service should provide analytics on the alias, for instance number of clicks and other data relevant to each shortened URL
• A user can have multiple alias's redirecting to the same long URL

Non-Functional:

• Latency - Users should receive a shortened alias in a short amount of time they input the long URL. Also users that click on the shortened alias should be quickly redirected to the long URL.
• Fault tolerance - These shortened URLs could fail to redirect.
• Scalability - Should be able to handle large volume of URL shortening requests and redirect efficiently
• High availability - Users at all times should be able to access their short URLs
• Partition Tolerance - In the event that a component in our system is down, users should still be able to access whatever information they need.

Capacity estimation

• 1 million URL shortening requests per day
• Each URL is on avg 100 characters long
• Each shortened alias is on avg 8 characters long
• We would need to store
• 100 characters per long URL
• 8 character per shortened alias
• 50 characters per metadata entry
• 20 characters per analytic entry
• 178 characters per entry
• # of requests per day * storage per entry
• 1,000,000 * 178 = 178,000,000 characters per day

API design

We would want a couple different API endpoints for this system

• Shortening URL endpoint
• POST '/shorten'
• Input: Long URL and an optional custom alias
• Output: Short URL
• URL Redirection endpoint
• GET '/short_code'
• Input: Short code
• Output: Redirection to the long code

Now we can also think about security for each short code, collision handling so that each short code is unique, expiration of each short code, rate limiting for the service and analytics.

Database design

Core Entities:

• Long URLs
• Short URLs
• Users
• Metadata

Relationships;

1. Long URLs are related to Short URLs with a one-to-many relationship
2. Users can have a one-to-many relationship to short URLs for custom aliases or many short URLs

Index:

• Create index on the short URLs column
• We can create an index on the timestamp for analytics and expiration

Scalability:

• We could create multiple copies of the same database behind a load balancer
• We could also shard the database

High-level design

1. Client layer
2. Frontend application - interface where users can input a long URL and receive a short URL. Sends requests to the backend API
3. Backend Layer
4. API Gateway - this would handle requests and send this to the appropriate service. Also handle authentication, and rate limiting
5. URL Shortening Service - this would handle generating and storing shortened URLs including the custom alias
6. Redirection Service - this would handle grabbing the long URLs from the database or cache and redirect users
7. Data Storage Layer
8. Primary DB - would store all short URLs, long URLs, user IDs, primary keys, timestamp. Could use MySQL or PostgreSQL
9. Caching Layer - this would be for frequently accessed short URLs, could use in-memory storage such as Redis, Amazon DynamoDB, Apache Cassandra
10. Supporting Services
11. Monitoring and Logging - Tools such as ELK stack, Grafana to monitor a systems health and log activities
12. Analytics Service - To track amount of clicks, geographical usage of clicks
13. Infrastructure Layer
14. Load Balancer - distribute incoming traffic across multiple instances of the backend services
15. Scalable Compute Resource - such as Kubernetes to manage and scale application instances
16. CDN - can be used to cache and quickly serve frequently accessed short URLs

Request flows

1. Client UI
2. User inputs a long URL and submits the form
3. The frontend sends an HTTP POST request to the '/shorten' endpoint of the API Gateway
4. API Gateway
5. Receives POST request
6. Performs checks, rate limiting, auth
7. Routes to the URL Shortening Service
8. Service
9. Parses the request to extract the long URL and any provided custom alias
10. If no custom alias, creates short code using hashing and encoding
11. Checks Redis cache to see if short code already exists
12. If exists, returns error
13. If not, proceeds
14. Stores created short code to the database along with a mapping to the long URL
15. Caches the short URL into Redis for future access
16. Returns the short code to the API Gateway
17. API Gateway
18. Forwards the short code to the frontend component
19. Client Response
20. Provides the short code and displays it to the user

Detailed component design

• Databases
• SQL databases such as MySQL or PostgreSQL
• Ensures strong consistency and durability for URL mappings
• Reliable support for transactions and indexing
• Redis for caching
• very fast read and write operations
• supports variety of data structures and operations
• can be used as both a cache and primary database

DB Schema:

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?