Architect a multi-tenant URL Shortening Engine

Question

Design a multi-tenant url shortening system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

### Functional Requirements
1. **URL Shortening**: Users can submit long URLs and receive a shorter version.
2. **Multi-tenancy**: Each tenant (e.g., different companies) should have isolated URL spaces.
3. **Analytics**: Track metrics such as click-through rates, geographic data of users, and device types.
4. **Redirection**: Users clicking on a shortened URL should be redirected to the original URL seamlessly.
5. **Authentication**: Ensure that only authorized users can access their respective URLs and analytics.
6. **API Access**: Provide a RESTful API for creating and retrieving shortened URLs and analytics.

### Non-Functional Requirements
1. **Scalability**: Handle millions of requests per day with the ability to scale horizontally.
2. **Availability**: The system should be up 99.99% of the time to support critical business operations.
3. **Latency**: URL redirection should occur in under 100ms.
4. **Security**: Ensure data privacy for each tenant and prevent unauthorized access.

Assuming Redfin's URL shortening service handles approximately 10 million requests per day:
1. **Requests per second (QPS)**: 10 million requests/day = ~115 QPS.
2. **Storage**: Assuming each URL mapping takes 256 bytes (URL + tenant ID + metadata), for 10 million URLs, total storage required = 10M * 256 bytes = ~2.5 GB.
3. **Bandwidth**: If each redirection request requires 1 KB of data (including headers), total bandwidth = ~115 QPS * 24 hours * 3600 seconds * 1 KB = ~10 TB/day.

### Component Diagram
- **Frontend**: A web interface for user interaction, built using React.
- **API Gateway**: To route requests to appropriate services (e.g., AWS API Gateway).
- **URL Shortening Service**: Handles creation and retrieval of shortened URLs (Node.js/Express).
- **Database**: A scalable NoSQL database like DynamoDB for storing URL mappings and tenant info.
- **Caching Layer**: Redis for caching frequently accessed shortened URL mappings.
- **Analytics Service**: Collect and process analytics data (Apache Kafka + Apache Spark).
- **Load Balancer**: NGINX or AWS Elastic Load Balancer for distributing traffic evenly across instances.
- **Monitoring**: Use Prometheus and Grafana for real-time monitoring and alerting.

### Schema Design
- **URLs Table**:  
  - `tenant_id` (Partition Key)  
  - `short_url` (Sort Key)  
  - `long_url` (String)  
  - `created_at` (Timestamp)  
  - `clicks` (Integer)  
  - `metadata` (JSON)
- **Analytics Table**:  
  - `tenant_id` (Partition Key)  
  - `short_url` (Sort Key)  
  - `timestamp` (Timestamp)  
  - `referrer` (String)  
  - `user_agent` (String)  
  - `geo_info` (JSON)

This design allows for efficient lookups by tenant and short URL while supporting rich analytics.

1. **Consistency vs Availability**: Opt for eventual consistency in the caching layer to improve availability and performance, while ensuring strong consistency in the database for URL mapping.
2. **Caching Strategies**: Local caching improves performance but may lead to stale data; using a distributed cache (Redis) ensures data consistency across multiple instances.
3. **Database Choice**: NoSQL (DynamoDB) offers flexibility and scalability but may require complex queries; a relational database could provide better transactional support but may not scale as well with heavy loads.
4. **Sharding**: Sharding based on tenant ID helps distribute load but introduces complexity in cross-tenant queries; we must ensure efficient data retrieval without compromising performance.
5. **Failure Handling**: Implement retries and fallbacks for transient failures, with monitoring for alerting on failures to ensure high availability.

Architect a multi-tenant URL Shortening Engine

Redfin

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Functional Requirements

Capacity Estimation

Submit Your Answer

Redfin Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search