System requirements

Functional:

1. generate short URL against long URL
2. short URL should redirect on long URL
3. Short URL should be valid for lifetime
4. REST APIs for generating short URL and redirection on short URL

Non-Functional:

1. Service should be consistent and highly available
2. low latency

Traffic:

1. Assuming 200:1 read/write ratio
2. Assuming number of unique shortened links generated per month = 100 million
3. Number of unique shortened links generated per seconds = 100 million per month/ 30(days) * 24(hour) * 8600(seconds) = 40 URLS per second
4. With 200:1 read/write ratio , number of redirection per second = 40 URLS per second * 200 = 8000 URLS per second

Storage:

1. Assuming lifetime of this service to be 100 years then total number of objects = 100 million per month * 100 (years) * 12 (month) = 120 billion
2. storage where 1 object with 500 bytes = 120 billion * 500 bytes = 60 TB

Cache:

1. following pareto principle 80:20 (80 % request for 20%data)
2. 8000 redirect requests per second will get, per day requests = 8000 * 86400 = 700 million
3. 20% requests need to be cache = 0.2 * 700 million * 500 bytes = 70 GB

Estimates

1. Shortened URLS generated = 40/s
2. shortened URLS for 100 years = 120 billion
3. total URL request/redirection = 8000/s
4. total storage = 60 TB
5. total cache = 70 GB

Algorithm and REST API

POST

URL - domain/create

Request Body - {long_url:""}

api_key: unique customer key

custom_url: optional value

Response Body - short url with 200 response

GET

URL- domain/get/{short_url}

Response - 302

Algorithm:

URL Encoding:

base62 (a-z, A-Z, 0-9) -

62^5 - 918 million urls

62^6 - 56 billion URLS

62^7 - 3500 billion URLS

we need 120 billion URLS, so 7 characters per short URL

MD5 - generate random 128 bit hash value and take first 7 character and check if exist in db generate new one to avoid collision

Database schema:

username

email

long url

short url

created_at

Capacity estimation

Estimate the scale of the system you are going to design...

API design

Define what APIs are expected from the system...

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...

High-level design

You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?