Architect a low-latency Ad Serving Engine

Question

Design a low-latency ad serving system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

### Functional Requirements:
1. **Real-time Ad Serving**: The system must serve ads in under 100ms for a seamless user experience.
2. **Targeting and Personalization**: Serve ads based on user behavior, demographics, and context.
3. **Campaign Management**: Allow advertisers to create, manage, and analyze ad campaigns via a dashboard.
4. **Analytics and Reporting**: Provide real-time metrics on ad performance including impressions, clicks, and conversions.
5. **Multi-region Support**: The system should serve users globally, ensuring low latency across regions.

### Non-Functional Requirements:
1. **Scalability**: Must handle millions of requests per second and scale horizontally.
2. **Availability**: 99.99% uptime, ensuring ads are served even during peak loads or failures.
3. **Data Consistency**: Support eventual consistency for user profiles and ad inventory.
4. **Security**: Ensure data protection for user privacy and secure transactions.
5. **Observability**: Implement logging and monitoring for real-time insights into system health.

### Back-of-Envelope Calculations:
- **Daily Active Users**: Assume 50 million daily active users.
- **Ad Requests per User**: Assume an average of 20 ad requests per user per day.
- **Total Daily Requests**: 50 million users x 20 requests = 1 billion requests per day.
- **Peak Requests per Second**: Assuming peak load at 10% of daily requests, we have:
  - 1 billion requests/day ÷ 86,400 seconds/day = ~11,574 requests/second on average.
  - During peak, estimate ~5x increase = ~57,870 requests/second.
- **Data Storage**: Assume 1 ad impression per request, requiring storage for ad performance data and user engagement metrics. Each record might be ~1 KB, leading to:
  - 1 billion records/day x 1 KB = ~1 TB of data per day.

### Conclusion:
This translates to a need for a highly scalable architecture capable of processing tens of thousands of requests per second with a robust data storage solution.

### Component Diagram:
1. **Load Balancer**: Distributes incoming requests across multiple ad serving instances to ensure low latency and high availability. Technology: **Nginx** or **HAProxy**.
2. **Ad Serving Layer**: Stateless microservices that handle ad requests. Each service instance can handle user requests independently. Technology: **Golang** for performance.
3. **Cache Layer**: In-memory caching for frequently accessed data (e.g., user profiles, ad inventory). Technology: **Redis** or **Memcached**.
4. **Database Layer**: A combination of SQL (for relational data) and NoSQL (for high-volume ad performance data). Technologies: **PostgreSQL** for structured data and **Cassandra** for wide-column storage of ad impressions.
5. **Analytics Service**: Asynchronous processing of ad performance and user engagement metrics. Technology: **Apache Kafka** for event streaming and **Apache Spark** for real-time analytics.
6. **Monitoring and Alerting**: Tools for operational excellence. Technology: **Prometheus** for metrics, and **Grafana** for visualization.

### Schema Design:
1. **User Table**: Stores user profiles.
   - `user_id` (PK, UUID)
   - `demographics` (JSON)
   - `preferences` (JSON)

2. **Ad Inventory Table**: Stores ads.
   - `ad_id` (PK, UUID)
   - `campaign_id` (FK, UUID)
   - `content` (TEXT)
   - `targeting_criteria` (JSON)

3. **Campaign Table**: Stores ad campaigns.
   - `campaign_id` (PK, UUID)
   - `advertiser_id` (FK, UUID)
   - `budget` (FLOAT)
   - `start_time` (TIMESTAMP)
   - `end_time` (TIMESTAMP)

4. **Impressions Table**: Stores ad performance metrics.
   - `impression_id` (PK, UUID)
   - `ad_id` (FK, UUID)
   - `user_id` (FK, UUID)
   - `timestamp` (TIMESTAMP)
   - `clicked` (BOOLEAN)

### Access Patterns:
- Frequent reads for user profiles and ad inventory (cached).
- Batch writes for impressions and analytics, leveraging event streams.

### Architectural Decisions:
1. **Consistency vs. Availability**: Chose eventual consistency for ad inventory and user profiles to ensure high availability and performance (CAP theorem). This is acceptable as ad serving can tolerate slight delays in data propagation.
2. **Caching Strategy**: Decided on a two-tier caching mechanism (in-memory and disk cache) to reduce latency. The trade-off is increased complexity in cache invalidation and potential stale data.
3. **Data Model Complexity**: Used a hybrid database strategy (SQL + NoSQL) to balance structured querying with high-volume writes. The trade-off is operational complexity and potential consistency issues across data stores.
4. **Load Balancing**: Chose to implement a global load balancer with geo-routing to ensure requests are served from the nearest data center, trading off potential latency for users far from the primary data center.
5. **Deployment Strategy**: Adopted a phased rollout using feature flags for new features in the MVP, allowing iterative improvements based on real user feedback while minimizing risk.

Architect a low-latency Ad Serving Engine

HashiCorp

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Functional Requirements:

Capacity Estimation

Back-of-Envelope Calculations:

Submit Your Answer

HashiCorp Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search