Design Monitoring Infrastructure for global users

Question

Design a scalable monitoring system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

### Functional Requirements
1. **Real-time Monitoring**: Capture and aggregate metrics from various applications in real-time.
2. **User Dashboard**: Provide a user-friendly interface for global users to visualize metrics and alerts.
3. **Alerting System**: Notify users of significant events or anomalies based on predefined thresholds.
4. **Data Retention**: Store historical data for analysis and reporting for at least 90 days.
5. **API Access**: Expose a RESTful API for users to programmatically access their metrics.

### Non-Functional Requirements
1. **Scalability**: Must handle millions of requests per minute, with the ability to add new sources and destinations easily.
2. **Availability**: System should maintain 99.9% uptime, ensuring global access.
3. **Latency**: End-to-end processing time for data should be less than 2 seconds.
4. **Security**: Data encryption both in transit and at rest, with role-based access control for APIs.

Assuming we expect to handle 10 million requests per day:
- **Daily Requests**: 10,000,000 requests
- **Requests per Second (RPS)**: $10,000,000 / 86400 \approx 115.74 	ext{ RPS}$
- **Data Size**: If each request generates around 1 KB of data (metrics + metadata), total daily data volume is 10 GB.
- **Storage Requirement**: Over 30 days, we would need about 300 GB of storage for metrics alone.
- **Infrastructure Scaling**: To accommodate peaks, we should plan for a 3x buffer, leading to a required throughput of ~350 RPS during peak times.

### Component Diagram
- **Data Ingestion Layer**: Use **Kafka** for message queuing to handle spikes in incoming requests. This allows asynchronous processing.
- **Processing Layer**: A cluster of **Apache Flink** or **Apache Spark** jobs to process incoming data in real-time and aggregate metrics.
- **Storage Layer**: Use **Amazon S3** for raw data storage and **Amazon DynamoDB** for fast, scalable access to aggregated metrics.
- **User Interface Layer**: A web application built using **React** to visualize metrics and manage alerts.
- **API Layer**: A set of RESTful services developed with **Node.js** for interactions with the data and user dashboards.

### Schema Design
1. **Metrics Table (DynamoDB)**:  
   - **Partition Key**: `userId`  
   - **Sort Key**: `timestamp`  
   - **Attributes**: `metricType`, `metricValue`, `alertThreshold`, `status` (e.g., OK, ALERT)

2. **Alerts Table (DynamoDB)**:  
   - **Partition Key**: `userId`  
   - **Attributes**: `alertId`, `alertType`, `alertMessage`, `timestamp`, `resolved` (boolean)

This schema allows efficient access patterns for retrieving user-specific metrics and alerts based on time.

### Architectural Decisions
1. **Consistency vs. Availability**: Choosing **DynamoDB** allows eventual consistency for reads, which is acceptable for real-time monitoring, prioritizing availability and performance. 
2. **Data Processing**: Real-time processing with **Flink** vs. batch processing with **Spark**—we choose real-time for immediate insights but accept some complexity in managing state.
3. **Caching Strategies**: Implement **Redis** for caching frequently accessed metrics to reduce load on DynamoDB, accepting potential staleness in the cache for performance gains.
4. **Load Balancing**: Use **AWS Elastic Load Balancer** in front of APIs to distribute incoming requests efficiently, ensuring horizontal scalability as user volume grows.

Design Monitoring Infrastructure for global users

Anthropic

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Functional Requirements

Capacity Estimation

Submit Your Answer

Anthropic Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search