System requirements

Functional:

List functional requirements for the system (Ask the chat bot for hints if stuck.)...

• Web browsers only
• Shortening URLs
• Use deduplication
• Redirection to original URL
• Custom alias
• Has to be unique
• Only alphanumeric and has a character limit
• Avoid reserved keywords such as "admin"
• Doesn't matter about upper/lower case, should point to the same URL.
• Analytics of shortened URL
• Click count
• Where clicks come from
• Users geo location of click
• Device info
• Top performing URLs
• Conversion tracking - measures specific actions taken by users after clicking on URL
• Expiration on URLs
• API to programmatically shorten URLs
• User auth so users can manage their URLs
• Unregistered users - can't customize or set expiration dates
• Registered users
• QR code gen for shortened URLs

Non-Functional:

• Performance
• It should quickly be able to generate URLs even during high traffic times.
• Reliability
• It should have a high uptime and reliable in generating accurate URLs
• Scalability
• It should be able to scale well with a large increase in users or URLs, without compromising performance
• User experience
• The user interface should be intuitive and user friendly, allowing for easy and quick access of shortened URLs, and be accessible across all devices and browsers.

Capacity estimation

Estimate the scale of the system you are going to design...

• 100,000 DAU each generating 5 URLs per day on average
• 500,000 URLs per day
• A peak day may be something like 1.6mill URLs shortened

API design

Define what APIs are expected from the system...

• URL shortening API: Endpoint that allows developers to programmatically create a shortened URL by sending a POST request with a long url
• Redirection API: Endpoint the provides the long url with a given short URL
• Analytics API: Endpoints that provide data on top performing urls, click count, geo location, and conversions
• User management API: An api that handles, account creation, account updates, auth, accessing user data.

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...

• Database Schema
• Users (id, username, email)
• URL (id, longURL, shortURL, userId, expiration, QRCode)
• Analytics (id, urlID, clickCount, conversion, location, deviceInfo)
• Relationships
• Each user can have one or many urls.
• Each url can only have one user associated with it.
• Analytics are linked to a single url.

High-level design

You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...

flowchart LR

Client --> API_Layer

API_Layer --> URL_Service

URL_Service --> Cache_Layer

Cache_Layer --> URL_Service_DB

URL_Service --> Queue

Queue --> Shortening_Service

Shortening_Service --> URL_Service_DB

API_Layer --> Analytics_Service

API_Layer --> User_Management_Service

User_Management_Service --> User_DB

Analytics_Service --> Analytics_Service_DB

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

sequenceDiagram

participant User

participant API_Layer

participant URL_Service

participant Queue

participant Shortening_Service

participant URL_Service_DB

participant Cache_Layer

participant Client

User->>API_Layer: Request to shorten URL

API_Layer->>URL_Service: Send long URL

URL_Service->>Queue: Add request to queue

Queue-->>Shortening_Service: Task picked up

Shortening_Service->>URL_Service_DB: Store short URL and QR code

URL_Service_DB->>Cache_Layer: Cache short URL

Shortening_Service-->>Queue: Task finished

Queue->>URL_Service_DB: Retrieve short URL from DB

URL_Service_DB-->>API_Layer: Return short URL to API

API_Layer-->>User: Receive shortened URL

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

• For the URL service, we will handle large scale by using the queue. When we have heavy traffic days, we'll be able to just put messages in the queue to get processed later on, so we don't overwhelm our system. The user will be able to get a message that it's processing so they know that it's taking some time. We can scale horizontally by spinning up more service instances. We can use database sharding to handle the large amount of urls that will be stored, allowing us to scale as much as needed.
• For the analytics service, we'll store the click of each user. We'll have to have a separate table that will store the click count, so we're not adding up all the rows for a specific url each time. This will make our database more performant when fetching the count and top performing urls.

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

• I used a queue instead of just writing directly to the database on request to make sure our system is resilient to days with high traffic.
• I used a cache in the url service to also keep things performant and not overload our database too much. You could run into issues of stale data here but I'd rather have that than our database getting overwhelmed. We can mitigate stale data with a time to live cache strategy.

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

• One of the services goes down.
• A database goes down.
• High traffic day.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?

• One of the services goes down solution - We could handle this by gracefully degrading. This will allow the user to still use our site, but only use features from the services that are still running
• A database goes down solution - We'll replicate our data so we can just route to one of the secondary databases.
• High traffic day solution - Our queue should be able to handle bottlenecks with a bunch of traffic and keep our system from getting overwhelmed.