System requirements

Functional:

1. Generate short url to a given long url.
2. redirect to the original url when clicked to short url.
3. provide analytics of link usage like: no. of clicks.
4. provide expiration time when a link is no longer accessible.
5. provide feature to user to make their own custom short url.

Non-Functional:

1. High Availability(at least 99.9%).
2. Scalable(system should be able to handle millions of request daily).
3. low latency.
4. Durability.
5. Security to prevent phishing.

Capacity estimation

1. Avg. URL shortening request = 1,000,000 per day
2. Read : Write ratio = 100:1
3. Peak traffic = 10x the avg. request
4. Url length = 100 characters

Throughput:

1. avg. writes per second(WPS) = 1,000,000 / 86400 = 12
2. max WPS = 10 * 12 = 120
3. avg. RPS = 12 * 100 = 1200
4. max RPS = 120 * 10 = 12000

Storage Estimation:

1. short url = 7 char
2. original url = 100 char
3. creation date = 8 byte
4. expiration date = 8 byte
5. click count = 4 byte

total storage = 127 bytes

total url per year = 127 * 1000000 * 365 = 127,000,000 = 46 GB

Bandwidth:

Assuming HTTPs 301 request is about 500 bytes(including headers and all)

1. total bandwidth per day = 500 * 1000000 = 50 GB
2. peak bandwidth = 500 * 12000 = 6 mb/sec

Caching estimation:

1. since its read heavy application we could use caching.
2. we can use 80:20 rule where 20% of url will generate 80% traffic
3. Since we have 1 million writes per day, if we only cache 20% of the hot urls in a day, Total cache memory required = 1,000,000 * 0.2 * 127 Bytes = 25.4 MB.
4. Requests hitting the DB: 1,200 × 0.10 ≈ 120 RPS

API design

we will use REST API.

Endpoint: POST /shorten

This endpoint creates a new short URL for a given long URL.

sample request:

{

"original_url" : "....",

"custom_name" : "" //optional

"expiration_date" : "....", //optional

"user_id" : "...."

}

sample response:

{

"short_url" : "...",

"original_url" : "...",

"expiation_date" : "...",

"creation_date" : "..."

}

Endpoint: GET /{short_url_key}

This endpoint redirects the user to the original long URL.

Sample Response:

HTTP/1.1 301 Moved Permanently Location: https://www.example.com/some/very/long/url

Database design

A NoSQL database like DynamoDB or Cassandra is a better option due to their ability to efficiently handle billions of simple key-value lookups and provide high scalability and availability.

We would need two tables: one for storing url mappings and one for storing user related information.

url_mapping_table features : {url_id , user_id ,short_url ,long_url ,creation_date ,expiration_date ,click_count}

user_table : {user_id ,name ,password ,e_mail}

High-level design

Load Balancer: Distributes incoming requests across multiple application servers.

Application Servers: Handles incoming requests for shortening URLs and redirecting users.

URL Generation Service: Generates short URLs, handles custom aliases, and manages link expirations.

Redirection Service: Redirects the users to the original URL.

Database: Stores mappings between short URLs and long URLs.

Cache: Stores frequently accessed URL mappings for faster retrieval.

Analytics Service (optional): Tracks usage statistics like the number of clicks, geographic location, etc.

Request flows

Shortening URL Flow:

User sends a request to Load Balancer to shorten a URL.

Load Balancer forwards the request to an Application Server.

Application Server interacts with the URL Generation Service to generate a short URL.

URL Generation Service stores the mapping in the Database and returns the short URL.

Application Server sends the short URL back to the User via the Load Balancer.

Redirection flow:

• • User requests a short URL redirection from Load Balancer.
  • Load Balancer forwards the request to an Application Server, which contacts the Redirection Service.
  • Redirection Service checks the Cache for the original URL.
  • If found, it is returned from the cache; otherwise, the Database is queried.
  • The original URL is fetched from the Database and stored in the Cache for future requests.
  • Redirection Service sends the original URL back to the Application Server to Load Balancer and then to the User to perform the final redirection.

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?