System requirements

Functional:

1. Generate short url to a given long url.
2. redirect to the original url when clicked to short url.
3. provide analytics of link usage like: no. of clicks.
4. provide expiration time when a link is no longer accessible.
5. provide feature to user to make their own custom short url.

Non-Functional:

1. High Availability(at least 99.9%).
2. Scalable(system should be able to handle millions of request daily).
3. low latency.
4. Durability.
5. Security to prevent phishing.

Capacity estimation

1. Avg. URL shortening request = 1,000,000 per day
2. Read : Write ratio = 100:1
3. Peak traffic = 10x the avg. request
4. Url length = 100 characters

Throughput:

1. avg. writes per second(WPS) = 1,000,000 / 86400 = 12
2. max WPS = 10 * 12 = 120
3. avg. RPS = 12 * 100 = 1200
4. max RPS = 120 * 10 = 12000

Storage Estimation:

1. short url = 7 char
2. original url = 100 char
3. creation date = 8 byte
4. expiration date = 8 byte
5. click count = 4 byte

total storage = 127 bytes

total url per year = 127 * 1000000 * 365 = 127,000,000 = 46 GB

Bandwidth:

Assuming HTTPs 301 request is about 500 bytes(including headers and all)

1. total bandwidth per day = 500 * 1000000 = 50 GB
2. peak bandwidth = 500 * 12000 = 6 mb/sec

Caching estimation:

1. since its read heavy application we could use caching.
2. we can use 80:20 rule where 20% of url will generate 80% traffic
3. Since we have 1 million writes per day, if we only cache 20% of the hot urls in a day, Total cache memory required = 1,000,000 * 0.2 * 127 Bytes = 25.4 MB.
4. Requests hitting the DB: 1,200 × 0.10 ≈ 120 RPS

API design

we will use REST API.

Endpoint: POST /shorten

This endpoint creates a new short URL for a given long URL.

Endpoint: POST /shorten

This endpoint creates a new short URL for a given long URL.

Database design

A NoSQL database like DynamoDB or Cassandra is a better option due to their ability to efficiently handle billions of simple key-value lookups and provide high scalability and availability.

We would need two tables: one for storing url mappings and one for storing user related information.

url_mapping_table features : {url_id , user_id ,short_url ,long_url ,creation_date ,expiration_date ,click_count}

user_table : {user_id ,name ,password ,e_mail}

High-level design

Load Balancer: Distributes incoming requests across multiple application servers.

Application Servers: Handles incoming requests for shortening URLs and redirecting users.

URL Generation Service: Generates short URLs, handles custom aliases, and manages link expirations.

Redirection Service: Redirects the users to the original URL.

Database: Stores mappings between short URLs and long URLs.

Cache: Stores frequently accessed URL mappings for faster retrieval.

Analytics Service (optional): Tracks usage statistics like the number of clicks, geographic location, etc.

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?