System requirements

Functional:

List functional requirements for the system (Ask the chat bot for hints if stuck.)...

• Read a user input URL
• Generate a unique URL
• Point this new URL to the same IP/DNS as the original

Non-Functional:

List non-functional requirements for the system...

• Make sure an URL is valid: use regex validation client-side and sanity check server-side, actually pinging the URL and seeing its response
• Generate a new and unique URL. The simplest way would be using your own domain and having variable URL params, e.g. with a hash string
• Store these URLs to ensure consistency. Need: shortened URL, URL to redirect to (additional: user in order to perform CRUD operations, added date)

Capacity estimation

Estimate the scale of the system you are going to design...

• Definitely a widely used service in terms of total number of users
• However, it will be a read-heavy system as many URLs will be of set-it-and-forget-it type
• An estimation in terms of traffic: 10 reads for every 1 write and 25,000,000 daily users, meaning 250 million reads and 25 million new URLs per day

API design

Define what APIs are expected from the system...

• GET /url/:id -> gets a shortened URL. The id will be equal to the URL param, e.g.: id = ggHHgg, url = bit.ly/ggHHgg
• POST /url/:id -> adds a new shortened URL. Needs: user, URL to redirect to. Writes: user, URL to redirect to, shortened URL id, creation date
• PATCH /url/:id -> modifies the shortened URL. Can be useful for a premium-exclusive functionality
• DELETE /url/:id -> deletes a URL row from the db

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...

Since it's a relatively easy structure that won't suffer heavy changes in the future and that requires ACID principles to ensure consistency, a SQL db will be useful here. Won't need many complex joins.

Possible tables:

• URLs: user, date added, shortened URL, URL to redirect to
• Users: user, user id, premium/trial status, payment status
• Custom subdomains (for premium users)

Having a read-only db replication will also be useful, mainly to have quicker reads for URLs. This will be coupled with a master db, that will also accept writes. Should the master fail, a replication can be created from the slave.

The URL tables can be sharded by hash ids in different alphabetical groups.

High-level design

You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...

• Client
• The request will go to a CDN who will query the load balancer. For reads, can also handle caching for highly requested URLs. This, coupled with ensuring client's geographical vicinity, will vastly improve performance
• A load balancer will be needed due to the high number of requests to be handled. Will want to redirect traffic to the least busy servers
• Several servers handling different microservices, namely:
• A few servers, the largest ones, in order to handle reading and writing URLs
• User management (changing name, trialing premium, adding payment method, etc.)
• Payments: coupled with a payments system like Stripe
• DBs:
• URLs: sharded alphabetically by shortened URL hash
• Users
• Custom subdomains

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

• URL creation
• Client sends a POST request to the server in order to create a new shortened URL
• This request will be handled by a load balancer and sent to the least busy server
• This server will validate the URL by checking that it returns a valid code (e.g. not a 404):
• If valid: 1. generate a unique hash, making sure this hash isn't used elsewhere within the same domain, 2. save this to the db, 3. return the response to the user
• URL retrieving
• Client inputs a URL in the browser
• URL reaches our DNS
• DNS reaches our CDN/load balancer
• CDN/load balancer will send this request to the least busy server
• Server will read the shortened URL param and choose which db to query from. These dbs are sharded alphabetically
• Server will retrieve the actual URL and perform a redirect to that URL by sending this back to the client

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

• Load Balancer:
• Can be useful for horizontal scaling, especially when paired with a CDN
• Allows for high availability too, which should be the priority for this app
• Paired with a CDN it ensures: geographical vicinity with end user, caching

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

• A possible issue in the future could be sharded dbs getting too large and performing further splits might get messy

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?