System requirements

Functional:

• Shorten URL:the system should allow userst to submit

a long URL and receive a shortened URL

• Redirect: user should be able to access the original URL by visiting the shortened URL.
• Analytics: the system should provide users with statistics on the usage of their shortened URLs
• Custom aliases:user is able to create a shortened URL with a custom alias instead of a randomly generated one.
• Expiration : allow for URLs to have an expiration date,

after which they no longer work.

• User management:user is able to create account,log in, and mange their shortened URLs
• API Access:there should be an API that allows third-party applications to generate and retrieve shortened URLs.
• Link preview: When a user hovers over or clicks on the shortened URL, the system may provide a preview of the long URL.

Non-Functional:

• Scalability:The system should be able to handle an increasing number of requests as the user base grows. This might involve horizontal scaling and load balancing strategies
• Availability: The service should ensure high availability, aiming for an uptime of at least 99.9%.
• Performance: The system should respond to requests in a timely manner, ideally in less than a second for URL shortening and redirection.
• Security: Implement measures to prevent abuse, such as rate limiting and validation checks for URLs.
• Maintainability: : The system should be designed in a way that makes it easy to update, debug, and manage.
• Data Consistency: Ensure that the URL mappings are consistent across the database and cache.
• Logging and Monitoring:the system should have logging capabilities to track usage patterns, errors, and other performance metrics.

1. User Experience: the user interface should be intuitive, and the service should provide an API with clear documentation for developers.

Capacity estimation

• Total URLs Shortened = User Base * Average URLs per User
  • 1,000,000 users * 10 URLs = 10,000,000 URLs

• Total URL Accesses = Total URLs Shortened * Average Accesses per URL
  • 10,000,000 URLs * 100 accesses = 1,000,000,000 accesses

Storage Calculation:

In terms of database and storage:

• Assuming each shortened URL requires:
  • 20 bytes for the original URL
  • 10 bytes for the short code/alias
  • 10 bytes for metadata (e.g., creation date, user ID)

Total size per URL = 20 + 10 + 10 = 40 bytes

• Total Storage Required = Total URLs Shortened * Size per URL
  • 10,000,000 URLs * 40 bytes = 400,000,000 bytes or approximately 400 MB of storage

Bandwidth Estimation:

If we expect that each request to the URL shortening service would require:

• 1 KB of data (for the request and immediate response)
• Then for 1 billion accesses:
  • Total bandwidth = 1,000,000,000 accesses * 1 KB = 1,000,000,000 KB or approximately 1,000 TB of data transmitted.

API design

API Endpoints

1. Create Short URL
   • Endpoint: POST /shorten
   • Request Body:

• • Responses:
    • 201 Created:

1. Redirect Short URL
   • Endpoint: GET /{shortCode}
   • Responses:
     • 302 Found: Redirects to the long URL.
     • 404 Not Found: If the short code does not exist.
2. Get URL Analytics
   • Endpoint: GET /analytics/{shortCode}
   • Responses:
     • 200 OK:

1. Update Short URL
   • Endpoint: PUT /shorten/{shortCode}
   • Request Body:

• • Responses:
    • 200 OK:

• • • 404 Not Found: If the short code does not exist.

1. Delete Short URL
   • Endpoint: DELETE /shorten/{shortCode}
   • Responses:
     • 204 No Content: Successfully deleted.
     • 404 Not Found: If the short code does not exist.

1. Caching Layer

• Improvement: Introduce a caching layer (like Redis or Memcached) to store frequently accessed shortened URLs.
• Benefit: This can reduce the load on the database and improve response times for redirects.

2. Advanced Analytics

• Improvement: Expand analytics capabilities to include:
  • Geolocation data (where users are accessing the URLs from).
  • Data on device type or browser used.
  • Referral sources to understand where traffic is coming from.
• Benefit: Provides users with detailed insights, potentially increasing user engagement.

3. Rate Limiting

• Improvement: Implement rate limiting per user or IP address to prevent abuse.
• Benefit: Safeguards the service against potential DDoS attacks or spamming of URLs.

4. Custom Alias Suggestions

• Improvement: If a user chooses a custom alias that’s already taken, offer suggestions for alternative aliases.
• Benefit: Enhances user experience by helping users find available options quickly.

5. URL Validation

• Improvement: Validate URLs to ensure they are correctly formatted and potentially harmful URLs can be flagged or blocked.
• Benefit: Improves security and helps prevent users from shortening harmful or malicious links.

6. User Dashboard

• Improvement: Create a user dashboard where users can see their shortened URLs, analytics, and manage their aliases.
• Benefit: Improves user engagement and allows users to manage their shortened URLs more effectively.

7. Mobile Optimization

• Improvement: Optimize the API and website for mobile usage or provide a dedicated mobile app.
• Benefit: Increases accessibility and user engagement from mobile users.

8. Setting Expiration for URLs

• Improvement: Allow users to set expiration dates for their shortened URLs.
• Benefit: Helps manage storage costs and ensures that outdated links are removed.

9. API Versioning

• Improvement: Implement API versioning to manage changes over time without disrupting existing users.
• Benefit: Ensures backward compatibility and a smoother transition for users when updates or changes are made.

10. Improved Security Measures

• Improvement: Implement OAuth or API keys for user authentication on the API.
• Benefit: Enhances security by ensuring that only authorized users can access or modify their URLs.

Database design

Core Entities

1. URL Mapping: This entity will store the relationships between the long URLs and their shortened aliases.
2. Users: This entity will hold user information if user accounts are permitted.
3. Analytics: This can be a separate entity to track URL access statistics, or it can be integrated with URL Mapping.

Entity Relationship Diagram (ERD)

Here's a proposed ERD to illustrate the relationships between these entities

Description of Entities

1. USERS:
   • id: Unique identifier for each user.
   • username: For user identification.
   • email: For communication and account recovery.
   • password_hash: Security measure. Ensure passwords are hashed.
   • created_at: Timestamp of user registration.
2. URL_MAPPING:
   • id: Unique identifier for each URL mapping.
   • long_url: The original long URL that users want to shorten.
   • short_code: The generated short code that users can share.
   • user_id: Foreign key linking to the user who created the URL (if applicable).
   • created_at: When the URL mapping was created.
   • expires_at: Optional expiration date for the URL.
3. ANALYTICS:
   • id: Unique identifier for each log entry.
   • url_mapping_id: Foreign key linking back to the URL mapping.
   • access_time: When the shortened URL was accessed.
   • user_agent: Information on the device/browser accessing the URL.

Considerations

• Indexes: Consider adding indexes on the short_code and long_url in the URL_MAPPING table for faster lookups.
• Sharding: If scaling requires, you might consider sharding the database based on user or URL patterns.
• Data Retention: Consider policies for how long to retain analytics data, as it might grow significantly over time.

High-level design

1. User Interface (UI): This can be a web application or mobile app that provides users with the ability to shorten URLs and manage their accounts.
2. API Gateway: A single entry point for all API requests. It handles routing requests to the appropriate service, authentication, and rate limiting.
3. Application Server: The core logic for handling URL creation, redirection, and user management resides here. It will handle incoming requests from the API gateway.
4. Database:
   • Stores the URL mappings and user data.
   • Handles analytics data regarding URL accesses.
5. Caching Layer: A caching solution (like Redis) to store the most frequently accessed short URLs to improve retrieval speed.
6. Analytics Service: A separate service that tracks the usage of shortened URLs, potentially aggregating and storing analytics data.

High-Level Design Diagram

Here's a conceptual diagram to visualize the high-level components and their interactions:

Description of Components

1. User Interface (UI):
   • Allows users to enter long URLs and receive shortened versions.
   • Provides a dashboard for managing shortened URLs and viewing analytics.
2. API Gateway:
   • Routes incoming requests to the relevant services.
   • Handles authentication and authorization.
   • Implements rate limiting to prevent abuse.
3. Application Server:
   • Contains the business logic for generating short URLs and handling redirects.
   • Interacts with the database to store and retrieve URL mappings and user data.
   • Uses the caching layer to quickly respond to frequently accessed URLs.
4. Database:
   • Stores URL mappings, user accounts, and access analytics.
   • Ensures data consistency and durability.
5. Caching Layer:
   • Provides fast access to popular URLs, reducing the load on the database.
   • Temporarily holds data that is frequently requested.
6. Analytics Service:
   • Collects and processes data on URL usage (e.g., clicks, geographic data).
   • Stores aggregated analytics in the database for user access.

Considerations

• Scalability: Ensure the architecture can scale horizontally, particularly the application server and caching layer.
• Security: Implement security measures for API access and data protection, especially for user information.
• Monitoring and Logging: Include systems for monitoring the performance of the services and logging relevant events for debugging and analytics.

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?