System requirements

Functional:

• Generate a unique short URL for a given long URL
• Redirect users to the original URL when they access the short URL
• Allow users to create custom short URLs (optional)
• Provide basic analytics for each short URL (e.g., click count)
• Allow users to set expiration dates for URLs (optional)
• Provide an API for programmatic access

Non-Functional:

• High availability (99.99% uptime)
• Low latency (redirect within 100ms)
• Scalability to handle millions of URL shortening requests per day
• Data consistency and durability
• Security to prevent abuse and malicious URL submissions

Capacity estimation

Assuming:

• 100 million new URL shortenings per month
• 10 billion redirects per month
• Average URL length: 100 bytes
• Short code length: 7 bytes

Storage estimation:

• New URLs per month: 100 million * (100 + 7) bytes ≈ 10.7 GB/month
• Annual storage growth: 10.7 GB * 12 ≈ 128.4 GB/year

Bandwidth estimation:

• Incoming: 100 million * 100 bytes ≈ 10 GB/month
• Outgoing: 10 billion * 7 bytes ≈ 70 GB/month
• Total: 80 GB/month

API design

1. Shorten URL

  Endpoint: POST /api/v1/shorten

  Request: {

   "url": "https://www.example.com/very/long/url",

   "custom_alias": "myurl" (optional),

   "expiry_date": "2024-12-31" (optional)

  }

  Response: {

   "short_url": "http://tny.io/abcd123",

   "expiry_date": "2024-12-31"

  }

2. Redirect

  Endpoint: GET /{short_code}

  Response: HTTP 301 redirect to original URL

3. Get URL Analytics

  Endpoint: GET /api/v1/analytics/{short_code}

  Response: {

   "short_url": "http://tny.io/abcd123",

   "original_url": "https://www.example.com/very/long/url",

   "creation_date": "2023-07-26",

   "expiry_date": "2024-12-31",

   "total_clicks": 1500

  }

Database design

Table: urls

• id: bigint (primary key)
• original_url: varchar(2048)
• short_code: varchar(7) (unique index)
• user_id: bigint (foreign key, optional)
• created_at: timestamp
• expires_at: timestamp
• click_count: int

Table: users (optional)

• id: bigint (primary key)
• email: varchar(255) (unique)
• password_hash: varchar(255)
• created_at: timestamp

High-level design

Frontend: Interface for users to interact with the service.

Backend: API to handle URL shortening, redirection, and analytics.

Database: Storage for URL mappings and analytics data.

Cache: Use a caching layer (e.g., Redis) to store frequently accessed data for faster redirection.

Load Balancer: Distribute incoming traffic across multiple backend servers

Request flows

URL Shortening:

1. Client sends POST request to /api/v1/shorten
2. Load balancer routes request to a web server
3. Web server forwards request to an application server
4. Application server generates a unique short code
5. Application server checks cache for existing short code
6. If not in cache, check database for uniqueness
7. Store new URL mapping in database and cache
8. Return short URL to client

URL Redirection:

1. Client sends GET request to /{short_code}
2. Load balancer routes request to a web server
3. Web server forwards request to an application server
4. Application server checks cache for short code
5. If not in cache, query database
6. Increment click count in database (asynchronously)
7. Return HTTP 301 redirect to original URL

Detailed component design

Load Balancer: Use a software load balancer like HAProxy or NGINX

Web Servers: Lightweight web servers like NGINX to handle HTTP requests

Application Servers: Use a language with good concurrency support (e.g., Go, Node.js)

Cache: Redis for fast key-value storage of URL mappings

Database: Use a relational database like PostgreSQL for primary storage

URL Shortening Algorithm: Base62 encoding of an auto-incrementing ID or random string generation

Analytics Service: Separate microservice for processing and storing click data

Trade offs/Tech choices

• Relational vs. NoSQL database: Chose relational for ACID properties and simpler querying
• Caching strategy: Cache-aside pattern for simplicity and eventual consistency
• Synchronous vs. asynchronous analytics updates: Chose asynchronous for better performance
• Custom short codes: Allow but with rate limiting to prevent abuse

Failure scenarios/bottlenecks

• Database failure: Use replication and failover mechanisms
• Cache failure: Fall back to database queries, with gradual cache rebuild
• Application server failure: Load balancer will route traffic to healthy instances
• Rate limiting: Implement at the application level to prevent abuse
• Database bottleneck: Implement read replicas and consider sharding for write-heavy loads

Future improvements

• Implement a Content Delivery Network (CDN) for faster global redirects
• Add more detailed analytics (e.g., geolocation, referrer data)
• Implement A/B testing capabilities for marketing use cases
• Develop mobile apps for URL management
• Add support for QR code generation
• Implement machine learning for detecting and preventing malicious URLs
• Expand API capabilities for enterprise integrations