Architect a multi-tenant URL Shortening Engine

Question

Design a multi-tenant url shortening system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

### Functional Requirements
1. **URL Shortening**: Users can submit a long URL and receive a shortened version.
2. **Redirection**: When a user accesses the shortened URL, they should be redirected to the original URL.
3. **Analytics**: Track metrics such as click counts, referrer information, and geographic data for each shortened URL.
4. **Multi-tenancy**: Support multiple users/organizations with their own individual dashboards to manage and analyze their shortened URLs.
5. **API Access**: Provide a RESTful API for URL shortening, management, and analytics.

### Non-Functional Requirements
1. **Scalability**: The system should handle millions of requests per day with low latency.
2. **Availability**: Aim for 99.99% uptime, ensuring the service is reliable and accessible.
3. **Performance**: Redirection should occur within milliseconds.
4. **Security**: Ensure user authentication and authorization for access to their shortened URLs and analytics.
5. **Monitoring**: Implement logging and alerting for tracking usage patterns and system health.

Assuming the system needs to handle **10 million requests per day**:
- **Average requests per second (RPS)**: 10 million requests / 86400 seconds = ~115 RPS.
- **Peak traffic estimation**: Assume peak traffic is 5x average, so ~575 RPS.
- **Data storage**: If each URL has an average size of 100 bytes (including metadata), and we estimate 1 million URLs in the database:
    - Total storage = 1 million URLs * 100 bytes = ~100 MB.
- **Scaling needs**: Design for at least 10 million URLs and 1 billion clicks, which requires around 10 GB for URL storage and additional storage for click data.

### Component Diagram
- **Load Balancer**: Distributes incoming requests to multiple application servers.
- **Application Servers**: Handle URL shortening and redirection logic, built using Node.js for performance and easy scalability.
- **Database**: Use PostgreSQL for structured data, including URL mappings and user analytics. Implement partitioning to manage large datasets.
- **Caching Layer**: Use Redis to cache frequently accessed shortened URLs and their corresponding long URLs to reduce database load.
- **Analytics Service**: A separate microservice that processes click data and aggregates it for reporting.
- **Monitoring**: Use Prometheus for metrics collection and Grafana for visualization, along with ELK stack for logging.

### Schema Design
1. **URLs Table**: 
   - `id SERIAL PRIMARY KEY`
   - `tenant_id UUID NOT NULL`
   - `short_url VARCHAR(10) UNIQUE NOT NULL`
   - `long_url TEXT NOT NULL`
   - `created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP`
   - `expires_at TIMESTAMP`

2. **Analytics Table**: 
   - `id SERIAL PRIMARY KEY`
   - `url_id INT REFERENCES URLs(id)`
   - `click_timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP`
   - `referrer VARCHAR(255)`
   - `user_agent VARCHAR(255)`
   - `geo_location VARCHAR(100)`

### Access Patterns
- Shortening a URL (write to URLs table).
- Redirecting (read from URLs table and log to Analytics table).
- Fetching analytics (read from Analytics table).

1. **Consistency vs. Availability**: Using eventual consistency for analytics data allows for higher availability but may delay reporting metrics slightly.
2. **Caching**: Storing the most accessed URLs in Redis speeds up redirection but introduces complexity in cache invalidation strategies.
3. **Database Choice**: PostgreSQL offers rich querying capabilities but may not scale as easily as NoSQL solutions like DynamoDB. The choice depends on the expected complexity of queries.
4. **Microservices vs. Monolithic**: A microservice architecture allows for independent scaling of components (e.g., analytics) but complicates deployments and inter-service communications.
5. **Rate Limiting**: Implementing rate limits at the API level can prevent abuse but may impact user experience for legitimate users, requiring a balance in thresholds.

Architect a multi-tenant URL Shortening Engine

Slack

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Functional Requirements

Capacity Estimation

Submit Your Answer

Slack Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search