System requirements

Functional:

1. URL Shortening: take a long URL and generate a unique short URL back
2. URL Redirection: When a short URL is accessed, it should redirect to the original long URL.
3. Custom URL Option: Users may want to create a short URL with a specific alias (optional).
4. Analytics: The service might track how many times a short URL has been
5. Expiration: The ability to set an expiration date for short URLs (optional feature).
6. User Management: If the service requires user accounts, functionality for registration, login, and managing user URLs.
7. Error Handling: Provide user-friendly error messages for invalid URLs or when a short URL does not exist.

Non-Functional:

1. Performance: The service should be able to handle a significant number of requests per second. For example, you might aim for 1000+ requests per second without noticeable latency.
2. Scalability: The system should be designed to scale horizontally to accommodate increasing amounts of traffic and data.
3. Reliability: The service should be highly available, with minimal downtime. An uptime of 99.9% or higher is often targeted.
4. Security: Implement measures to prevent malicious links and protect user data. HTTPS should be enforced to protect the integrity of the URLs being accessed.
5. Maintainability: The architecture should be designed in a way that allows for easy updates and maintenance without significant impact on service availability.
6. Usability: The user interface should be intuitive, making it easy for users to shorten URLs and manage their links.
7. Data Consistency: Ensure data integrity and consistency, especially when users are creating short URLs and those URLs are stored in a database.
8. Cost-efficiency: The system should be designed to minimize operational costs while providing the required functionalities.

Capacity estimation

Assume we have 1m shortening requests per day, let's assume 100:1 read/write ratio,

Traffic:

1m / 24 / 60 / 60 = 11.5 shortening requests per sec, and 100 * 11.5 = 1150 redirections per sec

Storage:

Assuming every request takes 1KB bandwidth, and we need to store 5 years data, this will take

1KB * 11.5 * 60 * 60 * 24 * 365 * 5 = 1729GB to store all data

Bandwidth:

For shortening, we assume 11.5 requests per second, so we need

11.5 * 1KB = 11.5KB/s

for read, 11.5 * 100 = 1150KB/s about 1MB/s

Memory: If we want to cache some of the hot URLs and follow 80-20 rule, meaning 20% urls get 80% traffic, we need

11.5 * 3600 * 24 =970MB, 970 * 0.2 = 194MB memory

API design

POST createURL(apiDevKey, originalURL, expireDate=None, customAlias=None, userName=None)

GET getURL(apiDevKey, shortURL)

PATCH deleteURL(apiDevKey, shortURL)

Database design

User table

UserID PK

Name

Email

CreationDate

LastLogin

URL table

Hash PK

OriginalURL

CreationDate

ExpirationDate

UserID FK

Since we are anticipating storing millions of URL objects and there is no relations between those URLs, we could use NoSQL store like DynamoDB or Cassandra.

High-level design

Components:

Metadata storage: SQL store

Object store: DynamoDB

Application: URL shortening application server

Key generation server: to pre generate a uuid to avoid conflicts and improve system performance, this includes key generation service, key-DB.

load-balancer

Cache server: Redis

Cleanup service: we could run cron jobs to cleanup database in non-active time

Request flows

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?