Design a low-latency Analytics System

Question

Design a low-latency analytics system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

- **Functional Requirements**:
  - Handle millions of low-latency analytics requests per second.
  - Provide real-time data processing and reporting with sub-second response times.
  - Support complex queries and aggregations over large datasets.
  - Enable user-defined alerts and thresholds for specific metrics.

- **Non-Functional Requirements**:
  - **Performance**: Target latency under 50ms for 95th percentile of requests.
  - **Scalability**: Must scale horizontally to handle increasing request loads.
  - **Availability**: 99.99% uptime with multi-region support to mitigate failures.
  - **Consistency**: Prioritize eventual consistency for analytical queries to optimize performance.

Assuming Citadel handles around 1 million requests per second (RPS) at peak:
- Each request is expected to read/write approximately 1 KB of data.
- Daily requests: 1M RPS * 60 seconds * 60 minutes * 24 hours = 86.4 billion requests/day.
- Total data processed daily: 86.4 billion requests * 1 KB = ~81.6 TB/day.

With a 30-day retention policy:
- Total storage required: 81.6 TB/day * 30 days = ~2.4 PB.

This estimation informs the need for a distributed storage solution and a robust caching mechanism to optimize read performance.

The architecture consists of the following components:
- **Load Balancer**: Routes incoming requests to appropriate services using NGINX or AWS ALB.
- **API Gateway**: Manages API requests and throttling using AWS API Gateway.
- **Analytics Service**: Stateless microservices designed with Go or Node.js for fast processing.
- **Data Processing Layer**: Utilizes Apache Kafka for message queuing and stream processing via Apache Flink.
- **Data Storage**: NoSQL databases like Amazon DynamoDB for high availability and low latency, and Amazon S3 for long-term storage of raw data.
- **Caching**: Redis for in-memory caching of frequently accessed data to reduce load on databases.

The architecture must be deployed across multiple regions to ensure low-latency access globally.

The schema for the analytics system might include:
- **Users Table**: (user_id, name, email, created_at)
- **Metrics Table**: (metric_id, user_id, timestamp, value, metadata)
- **Alerts Table**: (alert_id, user_id, metric_id, threshold, notification_channel)

- **Access Patterns**:
  - Frequent reads for metrics aggregated by user_id and timestamp.
  - Writes primarily for incoming metrics data and alerts.

Using a partition key on user_id ensures even distribution across shards, helping with scalability.

1. **Consistency vs Availability**: Given the analytics nature, we prioritize availability with eventual consistency. Users can handle slight delays in data reflecting real-time metrics.

2. **Complex Queries vs Performance**: To support complex queries, we consider pre-aggregating data in DynamoDB to reduce computational load during query time at the expense of real-time accuracy.

3. **Synchronous vs Asynchronous Processing**: Using Kafka allows us to decouple data ingestion from processing, enhancing performance but introducing potential delays in data visibility.

4. **Deployment Strategy**: Rolling deployments using blue/green strategies will minimize downtime. Chaos engineering practices (e.g., introducing failure scenarios) will be implemented to ensure system resilience under stress.

5. **SLOs/SLIs**: Aim for a 99.99% SLO on request latency, with SLIs monitored through Prometheus and Grafana for real-time insights.

Design a low-latency Analytics System

Citadel

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Capacity Estimation

Submit Your Answer

Citadel Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search