Requirements


Functional Requirements

Core Features

  1. Generate a shortened URL from a long URL.
  2. Redirect users from the shortened URL to the original URL.
  3. Store URL mappings persistently.
  4. Track basic analytics (optional):
    • Click count
    • Creation timestamp
    • Last accessed timestamp

System Components

API Service

Technology Stack

  • Python
  • FastAPI
  • Alembic

Responsibilities

  • Create short URLs
  • Validate URLs
  • Redirect requests
  • Expose analytics endpoints

Database

Technology: PostgreSQL

Responsibilities

  • Store URL mappings
  • Store metadata and analytics

Schema Example

urls ----- id checksum original_url created_at click_count

Indexes

UNIQUE INDEX(checksum) INDEX(created_at)

Since retrieval is primarily by short code, a unique B-tree index is sufficient.

Cache Layer

Technology: Redis

Responsibilities

  • Cache frequently accessed URLs
  • Reduce database reads
  • Share cache across multiple application instances

Cache Flow

Request ↓ Redis ↓ (miss) PostgreSQL ↓ Redis Update

For an MVP, an in-memory cache could be enough. Redis becomes valuable once multiple API instances are deployed.

Reverse Proxy

Technology: Nginx

Responsibilities

  • SSL termination
  • Rate limiting
  • Load balancing
  • Request filtering
  • Compression (gzip/brotli)

Flow

Client ↓ Nginx ↓ FastAPI

Non-Functional Requirements

Performance

Low Latency

  • Redis caching for frequently accessed URLs.
  • Proper indexing on checksum.
  • Use connection pooling.
  • Avoid unnecessary joins.

Scalability

Horizontal Scaling

Nginx ↓ ----------------------- ↓ ↓ ↓ App-1 App-2 App-3 ↓ Redis ↓ PostgreSQL

Strategies:

  • Stateless application instances.
  • Shared Redis cache.
  • Kubernetes for auto-scaling (future).

Reliability

Current MVP:

App ↓ PostgreSQL

Future:

Primary DB ↓ Read Replica

Possible improvements:

  • Automated backups.
  • Point-in-time recovery.
  • Health checks.

Resilience

Strategies:

  • Multiple application replicas.
  • Kubernetes restart policies.
  • Readiness and liveness probes.
  • Graceful shutdown handling.
  • Circuit breakers and retry policies for external services.

Observability

Logging

  • Structured JSON logs.
  • Rotating log files.
  • Request and error logging.

Monitoring

Recommended tools:

  • Prometheus
  • Grafana

Metrics:

  • Request count
  • Latency
  • Error rate
  • Cache hit ratio

Distributed Tracing

  • OpenTelemetry
  • Jaeger or Tempo

Security

Application Layer

  • URL validation.
  • Input sanitization.
  • Rate limiting.
  • API authentication (if admin endpoints exist).

Infrastructure Layer

Nginx provides:

  • TLS/SSL termination.
  • Request filtering.
  • Rate limiting.

Additional recommendations:

  • Security headers.
  • Secrets management.
  • Database credential rotation.
  • HTTPS-only communication.

Future Scope

  • Custom aliases (myapp.com/openai)
  • URL expiration
  • QR code generation
  • User accounts
  • Click analytics dashboard
  • Geo-location analytics
  • PostgreSQL replicas
  • Kubernetes deployment
  • CDN for global redirection performance


Capacity Estimation

Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...




API Design

For URL shortening, we can use FastAPI REST APIs to expose endpoints that allow users to create shortened URLs, redirect users, and monitor service health.

The API service will handle:

  • URL validation
  • Checksum generation
  • Duplicate URL detection
  • Storing URL mappings
  • Generating shortened URLs
  • Redirecting users to the original URL

API Contract

1. Create Shortened URL

Endpoint

POST base_url/v1/shortenUrl

Request Payload

{ "url": "http://example.com/aaaaaaaaaaa" }

Processing Flow

User URL | v Generate checksum | v Check PostgreSQL | +------ Exists ------> Return existing shortened URL | +------ New ---------> Store URL mapping | v Return shortened URL

Response Payload

{ "statusCode": 20001, "data": { "shortenedUrl": "https://base_url/a8f3d2" }, "errorDetails": [] }

Where:

  • statusCode → Application-level response code.
  • data → Contains generated shortened URL.
  • errorDetails → Contains error information if request fails.

Example error response:

{ "statusCode": 50001, "data": null, "errorDetails": [ { "errorCode": "INVALID_URL", "details": "Provided URL format is invalid" } ] }

2. Redirect Shortened URL

Endpoint

GET base_url/{hash_id}

Where:

  • hash_id is the checksum generated for the original URL.
  • The checksum acts as the unique identifier for retrieving the URL mapping.

Example:

Request: https://base_url/a8f3d2 Flow: Client | v Redis Cache | Cache Miss | PostgreSQL Lookup using checksum | Retrieve original URL | 301/302 Redirect

Response:

HTTP 302 Redirect Location: https://example.com/aaaaaaaaaaa

3. Health Check Endpoint

Endpoint

GET base_url/health

Purpose:

  • Kubernetes health monitoring.
  • Verify application availability.
  • Used for readiness/liveness checks.

Example response:

{ "status": "UP" }

Rate Limiting

Rate limiting can be implemented at two levels:

1. Reverse Proxy Level (Recommended for Production)

Using Nginx:

Client | v Nginx | v FastAPI

Advantages:

  • Requests are rejected before reaching the application.
  • Protects backend resources.
  • Better for large-scale deployments.

Example:

100 requests/minute/IP

2. Application Level (Development / Fine-Grained Control)

FastAPI middleware-based rate limiting can be used for:

  • Local development.
  • Testing.
  • API-specific limits.

Example:

POST /shortenUrl 50 requests/minute/user

Additional Considerations

Duplicate URL Handling

Since checksum is used:

URL A | v checksum = abc123 URL B | v checksum = abc123

Before storing:

  1. Generate checksum.
  2. Check existing checksum.
  3. If checksum exists:
    • Same original URL → return existing shortened URL.
    • Different URL → regenerate checksum to avoid collision.

Future API Extensions

GET /v1/analytics/{hash_id}

Provides:

  • Click count
  • Creation timestamp
  • Last accessed timestamp
DELETE /v1/url/{hash_id}

For URL expiration/removal.

  • This API design keeps the service simple for MVP while allowing future scaling with Redis caching, Nginx rate limiting, PostgreSQL replication, and Kubernetes deployment.



High-Level Design

The URL shortening system consists of multiple components responsible for accepting URL creation requests, storing URL mappings, serving redirects with low latency, and maintaining reliability and scalability.

The main components are:

  1. Client
    • Sends requests to create shortened URLs.
    • Accesses shortened URLs for redirection.
  2. Reverse Proxy (Nginx)
    • Acts as the entry point for all requests.
    • Handles:
      • SSL/TLS termination
      • Rate limiting
      • Request filtering
      • Load balancing between application instances
      • Compression
  3. FastAPI Backend Service
    • Core business logic layer.
    • Responsibilities:
      • Generate checksum for URLs.
      • Validate incoming URLs.
      • Check existing URL mappings.
      • Store new URL mappings.
      • Retrieve original URLs for redirects.
      • Provide health and analytics endpoints.
  4. Redis Cache
    • Used to reduce database load.
    • Stores frequently accessed URL mappings.
  5. Flow:
Request | v Redis | Cache Miss | PostgreSQL | Update Redis
  1. PostgreSQL Database
    • Persistent storage layer.
    • Stores:
      • Checksum
      • Original URL
      • Creation timestamp
      • Last accessed timestamp
      • Click count
  2. Monitoring and Observability
    • Application logs.
    • Metrics collection.
    • Distributed tracing.

Tools:

  • Prometheus
  • Grafana
  • OpenTelemetry


Database Design

Define the data model. Identify the main entities, their attributes, and relationships. Consider the choice of database type (SQL vs NoSQL) and justify your decision based on access patterns...




Detailed Component Design

Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.