System requirements

Functional:

List functional requirements for the system (Ask the chat bot for hints if stuck.)...

• Given a URL, the user should be able to get a shortened URL
• The user should be able to set the expiration time of the URL, optionally.
• User should also be able to get analytical metrics for the shortened URL such as number of clicks, etc.
• By default the URL once generated should never expire (before a predetermined max expiration limit) unless an explicit expiration is set by the user.

Non-Functional:

List non-functional requirements for the system...

• System should be highly available. If the system goes down, all the URL redirection will go down with it.
• URL redirection should happen quickly with minimal latency.
• Shortened links shouldn't be predictable to make them more secure.

Capacity estimation

Estimate the scale of the system you are going to design...

Assumptions:

• 30 million new URLS generated every month
• Store URLS generated in the last 5 years.
• URL consists of the characters [a-zA-Z0-9], which amounts to 62 characters.
• Average length of the URL in terms of characters: 2048

• Let's set 15 as the length of the shortened URL. Additionally, we will need to have 2 more properties for every URL generated - the creation date time and the expiration limit in minutes. We will allow a maximum expiration time of 5 years (The maximum expiration time allowed is 5 years. (2628000 minutes)

URL Class Properties:

originalURL: 2048 bytes or approximately 2KB for 2048 characters

shortenedURL: 15 bytes for 15 characters

urlID: 8 bytes

creationDateTime: 8 bytes for 8 characters

expirationTimeInMinutes: 8 bytes for 8 characters.

noOfClicks: 8 bytes

Total size: (2048 + 15 + 8 + 8 + 8) * 30 * 10^6 * 5 * 12 = 3.7 TB

API design

Define what APIs are expected from the system...

POST /get-tiny-url: For uploading a long URL and getting a short URL as a response.

GET /get-long-url/{shortened_url}: Return the long URL for the shortened URL.

GET /get-url-stats/{url_id}: To get stats about URL such no of click events. Returns as response number of click events.

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...

The data flow will happen between 4 components: User, Webservers and the database. There will also be a load balancer to handle high traffic of requests between the user and webservers and the webservers and database server. The system will also consist of cache to fetch long URLs of shortened URLs quickly.

User will prompt the interface with a long URL. The load balancer will decide which webserver to use to handle the request and then the webserver will forward the shortened URL to database server for storage. When fetching the long URL using the shortened URL, the webserver will first check the cache and if the long URL is not present, cache server will fetch it from the database. Accordingly the cache will get updated once fetched from the database. (per the read-through cache implemented).

High-level design

You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...

Primarily there are 5 components in this system: Client, Load Balancers, Database Servers, Cache servers and Key generation service.

• Client: The user using the URL shortening service.
• Load Balancers: This will be a program or third-party service such as AWS Load balancers, that will handle requests by client.
• Webserver: Main application server using the Key generation service to provide shortened URL.
• Key generation service: The service that will generate a shortened URL for a given long URL.
• Database servers: Main databases that will store the URL and their corresponding shortened URLs.
• Cache servers: Redundant storage for high and speedy availability.

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

There are 2 use cases in this application:

• Getting a shortened URL
  • Client sends a long URL with a request for shortening it.
  • Load balancer will forward the request to an available webserver. The webserver employes a key generation service to generate the shortened URL and sends it to database for storage.
  • Database stores the shortened URL and sends response confirming it.
• Accessing the shortened URL
  • Client sends a short URL to access the original long URL.
  • Load balancer forwards it to an available webserver.
  • The webserver will first try to get the long URL from cache if it exists there.
  • If the long URL is not in the cache, the cache server fetches it from the database, updates the cache and returns it to the webserver.
  • The webserver will now redirect to the long URL.

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

Load balancer: Uses a program or load balancing service like one by AWS so that's easily scalable and integrated.

Cache server: Uses a read-through cache instead of write-through because short URL is unlikely to change once created. There's no need for periodic updates in cache from the database server.

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?