Design Task Scheduling Infrastructure for real-time analytics

Question

Design a low-latency task scheduling system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

### Functional Requirements
1. **Task Scheduling**: The system should allow users to submit, update, and cancel tasks in real-time.
2. **Prioritization**: Tasks should be prioritized based on predefined rules or user input.
3. **Monitoring and Alerts**: Admins should receive alerts when tasks fail or exceed expected execution times.
4. **Task Dependencies**: Support for defining dependencies between tasks to ensure execution order.

### Non-Functional Requirements
1. **Low Latency**: The system should handle requests with a latency of less than 100ms.
2. **Scalability**: The system must handle millions of requests per second (QPS) efficiently.
3. **High Availability**: 99.99% uptime to ensure continuous task processing.
4. **Fault Tolerance**: The system must recover gracefully from failures without data loss.

To estimate capacity:
- **User Base**: Assume 10,000 concurrent users.
- **Requests per User**: Each user will generate an average of 10 tasks per second, leading to:
  - Total QPS = 10,000 users * 10 tasks/user = 100,000 tasks/second.
- **Data Storage**: If each task requires 1 KB of storage, for 1 million tasks, we would need:
  - Total Storage = 1,000,000 tasks * 1 KB/task = 1 GB.
- **Bandwidth**: Assuming each task communication (submit, updates, etc.) is around 500 bytes, for 100,000 requests:
  - Total Bandwidth = 100,000 tasks * 500 bytes = 50 MB/sec.

The architecture will consist of the following components:
1. **API Gateway**: AWS API Gateway to handle incoming requests, rate limiting, and security.
2. **Load Balancer**: AWS Elastic Load Balancer to distribute requests across multiple service instances.
3. **Task Scheduler Service**: Node.js application deployed on AWS ECS, responsible for managing task lifecycle and scheduling.
4. **Task Queue**: Apache Kafka for reliable task queuing and processing.
5. **Database**: PostgreSQL for storing task metadata and status, with read replicas for scaling read operations.
6. **Monitoring Service**: Prometheus and Grafana for real-time monitoring and alerting on task performance.

The database schema will include the following tables:
1. **Tasks Table**: 
   - `id`: UUID (Primary Key)
   - `user_id`: UUID (Foreign Key)
   - `status`: ENUM('pending', 'running', 'completed', 'failed')
   - `priority`: INTEGER
   - `created_at`: TIMESTAMP
   - `updated_at`: TIMESTAMP
   - `dependencies`: ARRAY of UUIDs (for task dependencies)
2. **Users Table**: 
   - `id`: UUID (Primary Key)
   - `name`: STRING
   - `email`: STRING
   - `created_at`: TIMESTAMP

### Access Patterns
- Frequent reads for task status updates and monitoring.
- Writes for task submissions and updates.

1. **Consistency vs. Availability**: Choosing eventual consistency for task updates to ensure high availability and low latency, with potential for stale reads.
2. **Database Selection**: PostgreSQL provides strong consistency guarantees but may introduce latency under heavy loads. Alternative NoSQL options like Cassandra could be considered for more write-heavy workloads at the cost of complex querying.
3. **Task Queuing**: Using Kafka introduces durability and fault tolerance but may add latency in task processing. We need to balance between guaranteed delivery and speed of processing.
4. **Monitoring vs. Performance**: Extensive logging and monitoring can impact performance; hence, we need to prioritize critical metrics and use sampling techniques.

Design Task Scheduling Infrastructure for real-time analytics

Goldman Sachs

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Functional Requirements

Capacity Estimation

Submit Your Answer

Goldman Sachs Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search