System requirements

Functional:

• Shorten URL:the system should allow userst to submit

a long URL and receive a shortened URL

• Redirect: user should be able to access the original URL by visiting the shortened URL.
• Analytics: the system should provide users with statistics on the usage of their shortened URLs
• Custom aliases:user is able to create a shortened URL with a custom alias instead of a randomly generated one.
• Expiration : allow for URLs to have an expiration date,

after which they no longer work.

• User management:user is able to create account,log in, and mange their shortened URLs
• API Access:there should be an API that allows third-party applications to generate and retrieve shortened URLs.
• Link preview: When a user hovers over or clicks on the shortened URL, the system may provide a preview of the long URL.

Non-Functional:

• Scalability:The system should be able to handle an increasing number of requests as the user base grows. This might involve horizontal scaling and load balancing strategies
• Availability: The service should ensure high availability, aiming for an uptime of at least 99.9%.
• Performance: The system should respond to requests in a timely manner, ideally in less than a second for URL shortening and redirection.
• Security: Implement measures to prevent abuse, such as rate limiting and validation checks for URLs.
• Maintainability: : The system should be designed in a way that makes it easy to update, debug, and manage.
• Data Consistency: Ensure that the URL mappings are consistent across the database and cache.
• Logging and Monitoring:the system should have logging capabilities to track usage patterns, errors, and other performance metrics.

1. User Experience: the user interface should be intuitive, and the service should provide an API with clear documentation for developers.

Capacity estimation

• Total URLs Shortened = User Base * Average URLs per User
  • 1,000,000 users * 10 URLs = 10,000,000 URLs

• Total URL Accesses = Total URLs Shortened * Average Accesses per URL
  • 10,000,000 URLs * 100 accesses = 1,000,000,000 accesses

Storage Calculation:

In terms of database and storage:

• Assuming each shortened URL requires:
  • 20 bytes for the original URL
  • 10 bytes for the short code/alias
  • 10 bytes for metadata (e.g., creation date, user ID)

Total size per URL = 20 + 10 + 10 = 40 bytes

• Total Storage Required = Total URLs Shortened * Size per URL
  • 10,000,000 URLs * 40 bytes = 400,000,000 bytes or approximately 400 MB of storage

Bandwidth Estimation:

If we expect that each request to the URL shortening service would require:

• 1 KB of data (for the request and immediate response)
• Then for 1 billion accesses:
  • Total bandwidth = 1,000,000,000 accesses * 1 KB = 1,000,000,000 KB or approximately 1,000 TB of data transmitted.

API design

API Endpoints

1. Create Short URL
   • Endpoint: POST /shorten
   • Request Body:

• • Responses:
    • 201 Created:

1. Redirect Short URL
   • Endpoint: GET /{shortCode}
   • Responses:
     • 302 Found: Redirects to the long URL.
     • 404 Not Found: If the short code does not exist.
2. Get URL Analytics
   • Endpoint: GET /analytics/{shortCode}
   • Responses:
     • 200 OK:

1. Update Short URL
   • Endpoint: PUT /shorten/{shortCode}
   • Request Body:

• • Responses:
    • 200 OK:

• • • 404 Not Found: If the short code does not exist.

1. Delete Short URL
   • Endpoint: DELETE /shorten/{shortCode}
   • Responses:
     • 204 No Content: Successfully deleted.
     • 404 Not Found: If the short code does not exist.

1. Caching Layer

• Improvement: Introduce a caching layer (like Redis or Memcached) to store frequently accessed shortened URLs.
• Benefit: This can reduce the load on the database and improve response times for redirects.

2. Advanced Analytics

• Improvement: Expand analytics capabilities to include:
  • Geolocation data (where users are accessing the URLs from).
  • Data on device type or browser used.
  • Referral sources to understand where traffic is coming from.
• Benefit: Provides users with detailed insights, potentially increasing user engagement.

3. Rate Limiting

• Improvement: Implement rate limiting per user or IP address to prevent abuse.
• Benefit: Safeguards the service against potential DDoS attacks or spamming of URLs.

4. Custom Alias Suggestions

• Improvement: If a user chooses a custom alias that’s already taken, offer suggestions for alternative aliases.
• Benefit: Enhances user experience by helping users find available options quickly.

5. URL Validation

• Improvement: Validate URLs to ensure they are correctly formatted and potentially harmful URLs can be flagged or blocked.
• Benefit: Improves security and helps prevent users from shortening harmful or malicious links.

6. User Dashboard

• Improvement: Create a user dashboard where users can see their shortened URLs, analytics, and manage their aliases.
• Benefit: Improves user engagement and allows users to manage their shortened URLs more effectively.

7. Mobile Optimization

• Improvement: Optimize the API and website for mobile usage or provide a dedicated mobile app.
• Benefit: Increases accessibility and user engagement from mobile users.

8. Setting Expiration for URLs

• Improvement: Allow users to set expiration dates for their shortened URLs.
• Benefit: Helps manage storage costs and ensures that outdated links are removed.

9. API Versioning

• Improvement: Implement API versioning to manage changes over time without disrupting existing users.
• Benefit: Ensures backward compatibility and a smoother transition for users when updates or changes are made.

10. Improved Security Measures

• Improvement: Implement OAuth or API keys for user authentication on the API.
• Benefit: Enhances security by ensuring that only authorized users can access or modify their URLs.

Database design

Core Entities

1. URL Mapping: This entity will store the relationships between the long URLs and their shortened aliases.
2. Users: This entity will hold user information if user accounts are permitted.
3. Analytics: This can be a separate entity to track URL access statistics, or it can be integrated with URL Mapping.

Entity Relationship Diagram (ERD)

Here's a proposed ERD to illustrate the relationships between these entities

Description of Entities

1. USERS:
   • id: Unique identifier for each user.
   • username: For user identification.
   • email: For communication and account recovery.
   • password_hash: Security measure. Ensure passwords are hashed.
   • created_at: Timestamp of user registration.
2. URL_MAPPING:
   • id: Unique identifier for each URL mapping.
   • long_url: The original long URL that users want to shorten.
   • short_code: The generated short code that users can share.
   • user_id: Foreign key linking to the user who created the URL (if applicable).
   • created_at: When the URL mapping was created.
   • expires_at: Optional expiration date for the URL.
3. ANALYTICS:
   • id: Unique identifier for each log entry.
   • url_mapping_id: Foreign key linking back to the URL mapping.
   • access_time: When the shortened URL was accessed.
   • user_agent: Information on the device/browser accessing the URL.

Considerations

• Indexes: Consider adding indexes on the short_code and long_url in the URL_MAPPING table for faster lookups.
• Sharding: If scaling requires, you might consider sharding the database based on user or URL patterns.
• Data Retention: Consider policies for how long to retain analytics data, as it might grow significantly over time.

High-level design

1. User Interface (UI): This can be a web application or mobile app that provides users with the ability to shorten URLs and manage their accounts.
2. API Gateway: A single entry point for all API requests. It handles routing requests to the appropriate service, authentication, and rate limiting.
3. Application Server: The core logic for handling URL creation, redirection, and user management resides here. It will handle incoming requests from the API gateway.
4. Database:
   • Stores the URL mappings and user data.
   • Handles analytics data regarding URL accesses.
5. Caching Layer: A caching solution (like Redis) to store the most frequently accessed short URLs to improve retrieval speed.
6. Analytics Service: A separate service that tracks the usage of shortened URLs, potentially aggregating and storing analytics data.

High-Level Design Diagram

Here's a conceptual diagram to visualize the high-level components and their interactions:

Description of Components

1. User Interface (UI):
   • Allows users to enter long URLs and receive shortened versions.
   • Provides a dashboard for managing shortened URLs and viewing analytics.
2. API Gateway:
   • Routes incoming requests to the relevant services.
   • Handles authentication and authorization.
   • Implements rate limiting to prevent abuse.
3. Application Server:
   • Contains the business logic for generating short URLs and handling redirects.
   • Interacts with the database to store and retrieve URL mappings and user data.
   • Uses the caching layer to quickly respond to frequently accessed URLs.
4. Database:
   • Stores URL mappings, user accounts, and access analytics.
   • Ensures data consistency and durability.
5. Caching Layer:
   • Provides fast access to popular URLs, reducing the load on the database.
   • Temporarily holds data that is frequently requested.
6. Analytics Service:
   • Collects and processes data on URL usage (e.g., clicks, geographic data).
   • Stores aggregated analytics in the database for user access.

Considerations

• Scalability: Ensure the architecture can scale horizontally, particularly the application server and caching layer.
• Security: Implement security measures for API access and data protection, especially for user information.
• Monitoring and Logging: Include systems for monitoring the performance of the services and logging relevant events for debugging and analytics.

Request flows

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?