Architect a low-latency Ad Serving Engine

Question

Design a low-latency ad serving system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

- **Functional Requirements**:
  - Serve targeted ads in real-time based on user behavior and contextual information.
  - Support ad formats including banners, video, and native ads.
  - Provide an API for ad request handling that includes rate limiting and authentication.
  - Enable reporting on ad performance metrics (impressions, clicks).

- **Non-Functional Requirements**:
  - **Latency**: Response time should be less than 100ms for 95% of ad requests.
  - **Scalability**: Handle up to 10 million requests per second during peak traffic.
  - **Availability**: 99.99% uptime with minimal downtime for maintenance.
  - **Security**: Ensure secure API access with OAuth 2.0 for authentication and validation.

- **Traffic Estimation**:
  - Assume HRT expects peak traffic of 10 million ad requests per second.
  - Each ad request generates a response of approximately 1KB.
  - Total data throughput: 10 million * 1KB = 10 TB per second.

- **Infrastructure Needs**:
  - Assuming each server can handle approximately 10,000 requests per second, we would need:
    - 10,000 servers to handle peak traffic.
  - Considering redundancy, we will provision for 20,000 servers to ensure high availability.

The high-level architecture includes the following components:

1. **API Gateway**: Handles incoming requests with rate limiting and authentication, using AWS API Gateway.
2. **Load Balancer**: Distributes requests to ad servers, using an ELB (Elastic Load Balancer) on AWS.
3. **Ad Serving Layer**: A cluster of Node.js servers for low-latency request handling, connected to:
   - **Cache Layer**: Redis for caching frequently served ads to minimize DB hits.
   - **Ad Database**: DynamoDB for scalable storage of ads and user profiles, providing eventual consistency.
4. **Analytics Pipeline**: A Kafka-based stream processing solution to handle real-time reporting and analytics on ad performance.
5. **Monitoring and Alerting**: Prometheus and Grafana for system monitoring.

The schema design to support access patterns includes:

- **Ads Table** (`AdID`, `Content`, `Format`, `TargetingCriteria`, `Price`, `Impressions`, `Clicks`)
- **User Profiles Table** (`UserID`, `Demographics`, `BehavioralData`, `AdPreferences`)
- **Impressions Log Table** (`LogID`, `AdID`, `UserID`, `Timestamp`, `ActionType`)

- **Access Patterns**:
  - Ads are fetched based on targeting criteria, making the `Ads Table` central. 
  - User profiles provide context for targeting, necessitating reads from the `User Profiles Table` to personalize ads.

1. **Consistency vs Availability**: Using DynamoDB allows for eventual consistency, which is sufficient for ad serving but may result in serving stale ads. This is acceptable given the nature of ad serving where latency is more critical.

2. **Caching Strategy**: While Redis offers low-latency responses, it introduces complexity in cache invalidation. We trade off some freshness for speed to ensure users receive ads quickly.

3. **Message Queuing**: Using Kafka for event-driven processing helps decouple the analytics workload from the ad serving workload, but introduces potential latency in reporting. This trade-off is manageable as real-time performance is prioritized over immediate analytics.

4. **Scaling Strategies**: Horizontal scaling with additional servers helps meet increased traffic, but may lead to increased costs. We must balance between infrastructure costs and performance needs.

Architect a low-latency Ad Serving Engine

HRT

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Capacity Estimation

Submit Your Answer

HRT Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search