Design Task Scheduling Infrastructure for real-time analytics

Question

Design a high-throughput task scheduling system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

- **Functional Requirements:**  
  1. Ability to schedule and execute millions of tasks per second for real-time analytics.  
  2. Provide APIs for task submission, status tracking, and result retrieval.  
  3. Support task prioritization and dependency management.  
  4. Allow for retries and error handling for failed tasks.  
  5. Enable real-time monitoring and alerting on task performance metrics.

- **Non-Functional Requirements:**  
  1. **Scalability:** Must handle 10 million requests per second (QPS) with the ability to scale horizontally.  
  2. **Availability:** System should have 99.99% uptime, with quick recovery from failures.  
  3. **Latency:** Average task execution time should be under 100 milliseconds.  
  4. **Consistency:** Eventual consistency is acceptable for task results, but strong consistency is required for task states.

- **User Load:**  
  Assume 10 million tasks are submitted per second (QPS).

- **Storage Needs:**  
  Each task requires approximately 1 KB of storage for metadata (including task ID, status, and results).  
  - Total storage per second: 10 million tasks * 1 KB = 10 GB/s.  
  - For a month, this translates to: 10 GB/s * 60s * 60min * 24h * 30d = 25,920 TB.

- **Network Bandwidth:**  
  Assume each task result is around 10 KB (after processing).  
  - Total outbound traffic per second for results: 10 million tasks * 10 KB = 100 GB/s.

![](https://example.com/component-diagram)  
- **Components:**  
  1. **API Gateway:** Handles incoming task requests, implements rate limiting and authentication.  
  2. **Task Scheduler:** A load-balanced service that distributes tasks to worker nodes.  
  3. **Worker Nodes:** Stateless microservices that execute tasks and store results.  
  4. **Message Queue (Kafka):** Ensures reliable task delivery and decouples task submission from execution.  
  5. **Database (Cassandra):** Stores task metadata and results, chosen for its high write throughput and scalability.  
  6. **Monitoring Service (Prometheus + Grafana):** Collects metrics and provides dashboards for real-time monitoring.

- **Task Table (Cassandra):**  
  | Column Name     | Type          | Description                         |  
  |------------------|---------------|-------------------------------------|  
  | task_id          | UUID          | Unique identifier for the task     |  
  | status           | TEXT          | Current state (pending, running, completed) |  
  | result           | BLOB          | Result of the task execution       |  
  | created_at       | TIMESTAMP     | Timestamp of task submission       |  
  | updated_at       | TIMESTAMP     | Last updated timestamp             |

- **Access Patterns:**  
  - Frequent writes on task submission and updates to status.  
  - Read operations for task results are less frequent but should be fast.

- **Consistency vs. Availability:**  
  Given the requirement for high availability, we choose eventual consistency for task results, accepting that users may not see the latest state immediately.

- **Database Choice:**  
  Chose Cassandra for its ability to handle high write loads and horizontal scalability, trading off on complex queries and strong consistency.

- **Task Scheduling:**  
  Implemented a round-robin load balancing strategy for task distribution, which is simple but may not account for worker load, potentially leading to uneven task distribution.

- **Failure Handling:**  
  Introduced a retry mechanism for task failures with exponential backoff, accepting increased latency for higher reliability.

Design Task Scheduling Infrastructure for real-time analytics

Mastercard

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Capacity Estimation

Submit Your Answer

Mastercard Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search