System requirements

Functional:

List functional requirements for the system (Ask the chat bot for hints if stuck.)...

Non-Functional:

List non-functional requirements for the system...

Capacity estimation

Assuming 10: 1 read to write ratio.

Everyday we have 1M reads, this will give 30M reads per month and 360M reads per year. Assusming a shortened url will expire in 5 years, since we have 10: 1 ratio, so writes will be 36M. Now, assuming we will store 1Kb data per record, this will give us:

1KB * 36M * 5 = 180 GB of data stored

Also we will have 100 reads/sec

As we can see that the application is read heavy. So it is very essinitial to scale out as the requirement for the reading grows. And we know that a nosql db will be very useful in our case. Also we should replicate our data across multiple instances of DB and use the replicas only for reading the url and not the primary db. So that the load is distributed. So we will be using mongo db for storing the records with schema.

Also we will use redis as key value store for our caching as redis is fast and scalable

API design

POST: '/'

Database design

Following are the schemas that will be used:

URL table:

• Short url: url
• Original url: url
• created_at: date
• created_by: user
• expiration_time: Date

User table:

• UserId
• name
• email
• phone

High-level design

• Load balancer used for uniform distribution of load across servers
• services such as 'Cache Service', 'URL Generation service' and 'URL expiration service'.
• Cache service helps in caching most accessed URLs and providing them without accessing the db
• URL generation service takes the long url and creates the short url for the user
• URL expiration service is used for invalidating the URLs stored and invalidate them once their expiration_time has passsed

Request flows

• When a client sends a request, based on whether it's a write or read request, the request is routed to appropriate service
• If it's a request for creating a short url for the provided url, the request will be routed to url generation service which will the n return the shortened url
• If it's a request to get the original url for a short url, then it's routed to cache service. If it's a cache miss, the url is fetched from the replica db.

Detailed component design

1. URL Generation service: takes the original long url, and hashes it to give the short url. The short url is written to the primary db
2. Cache Service: Used for caching of urls, redis caching can be used to appropriately for this
3. URL expiration service: Used for removing the expired records. Runs a job every night at 12.

Trade offs/Tech choices

• Although mongoDB seems most appropriate for this case, but the eventual consistency issue with NoSQL DB might give stale results
• Although using replication might reduce the load on the primary db but again if replication is in progress, the read for the user might give stale results
• We are placing 'availability' over 'consistency'

Failure scenarios/bottlenecks

• If the URL generation service fails the whole application might go down as it will not be able to serve it's original purpose
• Also if the expiration service fails, our db might be flooded with records

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?