System requirements

Functional:

1. Convert long URL to short URL
2. On clicking the short URL it must direct to the destination same as long URL
3. Shortened URL can be customized
4. Tracking Click Analysis
5. URL Management - should be able to view the URL's shortened

Non-Functional:

1. Scalable
2. Available
3. Secure
4. Less Latency
5. Performance
6. Consistency

Capacity estimation

1. Number of users - 10,00,000 per month
2. 5 URLS per month - 5*10^6 URL creation requests per month
3. It has more reads than write - it is a read-heavy application
4. per day - 5*10^6/30 = 1600000/24 * 60 * 60 = 18 request per second
5. Storage estimation - 5 years, will we be retaining the shortened URL's?
6. Let's assume size of shortened URL, long URL, and userId will be 1KB, total storage - 18 KB*5*12*30*24*60*60 = 5 TB
7. Network Estimation = 18kb/s
8. Memory Estimation - Cache can be used for quick reading, and better performance - 20% of the URLs are stored as they are the hot spots - 1TB

API design

1. createShortenedURL(String userId, String longURL, String customizedUrl) - to create the short URL - by using hashing technique or algorithms
2. getShortenedUrls(String userId) - gets all shortened URLs to view.
3. getShortenedUrls(String userId, String longUrl) - gets all shortened URLs to view.

Database design

To store user details, and the URL details - We can use a relational database for storing user details and a no SQL DB for the key-value pair - Key the long URL and value will be the shortened URL - We can use Dynamo DB or Cassandra

High-level design

When the clients request for a tiny URL, it is sent to load balancers first to distribute the requests once the server is determined, the server creates a tiny URL using the creating service - which can use a hashing algorithm to create the tiny URL or takes in customizations and then stores it in NoSQL. If the client hits to get the existing tiny URL, it looks up into the cache, and if it doesn't find it will hit the database.

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

Detailed component design

1. Database components - we are using NoSQL and relational database to scale the NoSQL as the request grows, we can use vertical or horizontal scaling, but since vertical scaling can hit the limits, we can use horizontal scaling - by partitioning the data into shards, using hash-key sharding, since sorting might not be helpful here. To purge data after 5 years - we can use clean up jobs in the database
2. Cache - Since 20% of hotspots can be stored in Cache and its a read-heavy application, we can use read through cache strategy here - first read from cache is unable hit the database and the cache eviction policy can be least recently used element. The cache expiration policy cannot be too short or long.
3. Load Balancers can use round robin algorithm to assign the request to a server.

Trade offs/Tech choices

1. In horizontal scaling of databases - hash key partition can be a problem if most of the hotspots are present there, so better to have a shard for every hotspot.

Failure scenarios/bottlenecks

1. Hashing algorithm ig cannot generate different unique urls then the system could fail.

Future improvements

To come up with different algorithms like to append the digested value of user_id and to append it to the shortened URL