System requirements

Functional:

List functional requirements for the system (Ask the chat bot for hints if stuck.)...

1) Shorten URL : Take long URL and return shortened URL

2)Redirect: When short URL is called redirect to Long URL

3)Analytics: Provide analytics of number user clicks/redirects etc.

4)Expiry: User can define expiry of URL

5)API design: provide API for dev to integrate the URL shortening

Non-Functional:

1)Performance: System should handle thousand request per second, with less than 100ms delay

2)Scalability: The architecture should support horizontal scaling to accommodate increasing traffic

3)Reliability: The system should have 99.99% uptime.

4)Security: Implementing measures to prevent abuse such as rate limiting.

5)Data Integrity: Ensure that all shortened URLS point to correct long URLs.

6)User privacy: Avoid storing unnecessary user data

Capacity estimation

traffic : 1million user request per day

data volume of 1 request= 1kb

storage required for a day = 10^6 * 1kb= 1GB

Storage required over the year = 365GB

We can consider a 1TB monolithic server for the initial design

API design

REST API

1) POST /shorten-url/

Request Body

{

"longUrl":"asfafaffafafafafafasdf".

"expiryDateTime":"2024-10-31 10:15:00"

}

2)GET /get-url/{shorten-url}

Rate limit post url to prevent DDOS attack

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...

url_shortner_table

id

long_url

short_url

expiry_date_time

createdAt

createdBy

create a read replica for read request to be handled via sharding

High-level design

• Client: The end-user application or browser that makes HTTP requests to shorten URLs or access shortened URLs.
• Load Balancer: This component distributes incoming requests across multiple application servers to ensure no single server is overwhelmed, thus improving performance and reliability.
• Application Server: This handles the core logic of the URL shortening service, including shortening URLs, handling redirects, and serving analytics. It interacts with both the primary database and read replicas.
• Primary Database: This is where the main data is stored, such as original URLs and their corresponding shortened versions.
• Read Replica Database: Used for read-heavy operations, this improves performance by offloading some of the read requests from the primary database, ensuring quicker response times for users.
• Analytics Service: A dedicated service (or part of the application server) that handles logging and retrieving analytic data like click counts and user engagement details.
• Cache Layer: Often utilized to store frequently accessed data and reduce database load, providing faster response times.

Request flows

URL Shortening Request

1. Client Request: User sends a long URL to the server to shorten.
2. Unique Code Generation: Server generates a unique short code (random hash, base62 encoding).
3. Store Mapping: Store the short code-long URL mapping in the database.
4. Return Short URL: Server returns the short URL to the user.

URL Retrieval Request

1. Client Request: User requests the original URL by entering the short URL.
2. Database/Cache Check: Server first checks the cache for the short code. If not in the cache, it queries the database.
3. Redirect: Server redirects the user to the original long URL.

Detailed component design

URL Shortening Algorithm:

Use base62 encoding to create short codes with alphanumeric characters.

Cache:

• Use Redis to cache recently accessed URLs.
• Implement cache eviction (e.g., LRU) to manage storage.

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

1. Scalability
   • Vertical Scaling vs. Horizontal Scaling:
     • Vertical Scaling: Involves upgrading existing server resources (CPU/RAM), limited by the maximum capacity of the machine.
     • Horizontal Scaling: Involves adding more servers to distribute the load, facilitating better scalability but requiring load balancing and data synchronization.
   • Trade-off: Vertical scaling is simpler but limited, while horizontal scaling can handle more growth but requires more effort in management.
2. Load Balancer Choices
   • Layer 4 Load Balancer (Transport Layer): Handles TCP traffic, offering high performance, but doesn't have visibility into HTTP request content.
   • Layer 7 Load Balancer (Application Layer): Can make routing decisions based on HTTP request content, enabling functionalities like cookie-based session persistence.
   • Trade-off: Layer 7 provides more features at the cost of slightly increased latency compared to Layer 4.
3. Database Choice
   • SQL vs. NoSQL:
     • SQL (e.g., MySQL, PostgreSQL): Enforces data integrity and supports complex queries, but can be slower for write operations and scaling horizontally.
     • NoSQL (e.g., MongoDB, DynamoDB): Allows for horizontal scaling and can handle large amounts of unstructured data but may lack ACID compliance.
   • Trade-off: SQL provides better relational support but may require more effort to scale, while NoSQL offers scalability at the cost of transactional integrity.

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

1. Database Failures
   • Scenario: The primary database becomes unavailable due to server crashes, connectivity loss, or maintenance.
   • Bottleneck: All write operations would fail to complete, impacting URL shortening functionality.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?

Mitigation: Implement a failover mechanism with read replica databases or a high-availability setup. Utilize connection pooling and graceful degradation to ensure continued read operations.