System requirements

Functional:

List functional requirements for the system (Ask the chat bot for hints if stuck.)...

Input: long URL

Output: A shortened URL that redirects to . In order to keep popular URLs from overwhelming the system, a given input URL will always return the same output URL.

Stretch goal: Potentially identify variables in an input URL and add them to the end of the shortened URL. e.g. xyzwithlonname.com/lotsofurl/search?q=?q=

Non-Functional:

List non-functional requirements for the system...

Availability - we want this service to be available (e.g. return responses within 1 second) 99.9% of the time.

Performance - See above. We want the case of batched or automated requests to the system to be performant so a goal of 99% of requests in <=.1s and 99.9% of requests in <= 1s would be ideal. For a small task like checking the existence of a hash and shortening it, this should be feasible, although I am unfamiliar with networking and am not sure what bottlenecks exist there.

Scalability - estimate 10 requests per second, with a maximum batch size of 100 requests and an estimated maximum of 1,000 requests in a second.

Security - No PII in input URLs, presumably, but maybe LLM can check for that. Possible to safeguard against malicious URLS, although I'm not familiar enough with networking measures to achieve that.

Capacity estimation

Estimate the scale of the system you are going to design...

See scalability above: estimate 10 requests per second, with a maximum batch size of 100 requests per user per second and an estimated maximum of 1,000 requests in a second.

API design

Define what APIs are expected from the system...

POST shortenURL(long_url, expiration_time=None, custom_short_url=None)

• Input: long URL to be shortened
• Returns: a new shortened URL (if the input long URL has never been provided to the system) or the previously-generated short URL for that input.

GET redirectURL(short_url)

• Input: a previously shortened URL
• Returns: the long URL that was shortened to the input URL (if it exists)

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...

Requirements:

• Two fields; shortURL and longURL
• Quick existence lookup of both URLs, quick matching between URLS.
• Uniqueness of both fields.
• This could simply be done by indexing both long and short URLS. We could also use a deterministic hashing algorithm for URLS that would ensure that the same URL always provides the same shortened one. Another (probably the best option) is to use a unique sequential integer identifier, and use indexed DB lookups to check whether or not the longURL has already been encoded.

URLMapping {

VARCHAR shortURL PRIMARY KEY

VARCHAR longURL UNIQUE INDEX

timestamp createdAt

timestamp expiresAt

}

High-level design

You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...

FE API: exposes shortenURL and redirectURL methods via API.

Server: takes requests, looks up values in the database, and creates new shortURLS as necessary

Cache: LFU cache (e.g. Redis) stores most frequently used keys.

Database: stores shortURL and longURL table, shortURL is primary-indexed for efficient lookup.

TODO: How do I pick which database to use?

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

Client makes a request.

shortenURL(longURL):

• sends longURL to the server
• server checks whether or not longURL exists in the cache, then checks the database. if not in either, it generates a new sequential ID and enters it into the database.
• this SHOULD not cause collisions – however, if the server should still check for collisions when it creates a new key just in case of developer error. if it finds a collision, try adding a random alphabetical string to the sequential ID, which is almost guaranteed to be unique if the ID is in integer.
• note - sequential ID should be LOCKED to threads -- only one thread accessing it at a time.
• server returns the corresponding shortID

redirectURL(shortURL)

• sends shortURL to the server.
• server looks up shortURL in the DB. if it exists, returns a SUCCESS message and a URL to redirect to. if not, returns a FAILED message and returns a message stating that no such short URL exists.

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?