Design a Content Delivery Network

Question

Design a CDN for serving static and dynamic content globally. Cover cache hierarchy, cache invalidation strategies, origin shielding, edge compute, and handling cache stampedes during viral content spikes.

Codemia · Accepted Answer

### Functional Requirements:
1. **Content Distribution**: Serve static content (images, videos, scripts) and dynamic content (API responses) globally with low latency.
2. **Cache Hierarchy**: Implement a multi-tier cache hierarchy including edge caches and regional caches.
3. **Cache Invalidation**: Support efficient cache invalidation for dynamic content and allow for manual and automated invalidation strategies.
4. **Origin Shielding**: Protect the origin servers from spikes in traffic by directing requests through a designated shield layer.
5. **Real-time Analytics**: Provide real-time analytics on cache hit/miss ratios, user requests, and content performance.

### Non-functional Requirements:
1. **Scalability**: The system should be able to handle millions of concurrent users and scale horizontally.
2. **Availability**: Aim for 99.99% uptime with automatic failover capabilities.
3. **Performance**: Latency should be under 100ms for 95% of user requests globally.
4. **Security**: Implement DDoS protection and secure content delivery over HTTPS.

### Capacity Estimation:
Assuming Google serves content to 2 billion users with an average of 10 requests per user per day:
1. **Total Requests**: 2 billion users * 10 requests = 20 billion requests per day.
2. **Peak Traffic**: Assume peak traffic occurs during special events, leading to a 5x increase in requests. Therefore, peak requests = 20 billion * 5 = 100 billion requests/day.
3. **Requests per Second**: 100 billion requests/day / 86400 seconds = ~1.16 million requests per second.
4. **Storage Requirements**: If each static asset is 1 MB and we aim to cache 1 trillion assets, we need approximately 1 Exabyte of storage, factoring in redundancy and replication.

### High-level Architecture:
- **Components**:
  1. **Edge Nodes**: Deployed globally to cache content closer to users. Technologies: NGINX for HTTP caching, Varnish for caching dynamic content.
  2. **Regional Caches**: Regional data centers that hold a larger cache of popular content. Technologies: Redis for fast access to frequently requested data.
  3. **Origin Servers**: Centralized servers that hold the master copy of the content. Technologies: Google Cloud Storage for static assets, Google Compute Engine for dynamic content.
  4. **Load Balancer**: Distributes traffic across edge nodes. Technologies: Google Cloud Load Balancing.
  5. **Analytics Service**: Gathers and processes real-time data from edge and regional caches. Technologies: Google BigQuery for analytics processing.

![Component Diagram](https://example.com/diagram)  
*(This is a placeholder for a diagram that would typically illustrate the architecture.)*

### Data Model:
- **Content Table**:
  - `content_id` (Primary Key)
  - `content_type` (ENUM: image, video, script)
  - `url` (VARCHAR)
  - `cache_status` (ENUM: cached, invalidated, expired)
  - `timestamp` (DATETIME)

- **Cache Table**:
  - `cache_id` (Primary Key)
  - `content_id` (Foreign Key to Content Table)
  - `location` (VARCHAR)
  - `last_accessed` (DATETIME)
  - `expiry_time` (DATETIME)

- **Analytics Table**:
  - `analytics_id` (Primary Key)
  - `content_id` (Foreign Key)
  - `request_time` (DATETIME)
  - `response_time` (FLOAT)
  - `status_code` (INT)

This schema supports efficient querying for cache management and analytics.

### Key Trade-offs:
1. **Consistency vs. Availability**: In a global CDN, we may sacrifice strict consistency for higher availability and faster access times. Using eventual consistency for cache updates allows for a more responsive system.
2. **Cost vs. Performance**: Deploying more edge nodes increases costs but improves performance. We need to balance the number of edge nodes with anticipated traffic patterns.
3. **Cache Size vs. Hit Ratio**: Larger caches can hold more content but may lead to higher miss rates if the cache is not managed efficiently. Implementing algorithms like LRU (Least Recently Used) can help optimize cache usage.
4. **Real-time Analytics vs. Latency**: Providing real-time analytics can introduce latency in request processing. We might use sampling techniques to reduce the load on the analytics service while still gathering useful data.

Design a Content Delivery Network

Google

How to Approach This

Sharpen Your Skills on Codemia

Sample Answer

Requirements

Functional Requirements:

Capacity Estimation

Capacity Estimation:

Submit Your Answer

Google Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search