Build a low-latency Task Scheduling Pipeline

Question

Design a low-latency task scheduling system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

- **Functional Requirements**:
  - Users should be able to create, update, and delete tasks.
  - Tasks must be scheduled with specific execution times, with support for recurring tasks.
  - The system should support querying tasks based on user ID, status, and execution time.
  - Notifications should be sent upon task completion or failure.

- **Non-Functional Requirements**:
  - **Latency**: The system should handle requests with a latency of less than 100 ms.
  - **Throughput**: Capable of processing 10,000 requests per second.
  - **Scalability**: Must support horizontal scaling to manage increased load.
  - **Durability**: Task data must be persisted and recoverable in case of failures.
  - **Availability**: The system should have at least 99.9% uptime.

- **User Base**: Assume 1 million active users.
- **Requests Per User**: Each user generates about 10 requests per day.
- **Total Daily Requests**: 1,000,000 users * 10 requests = 10 million requests/day.
- **Peak QPS**: Assuming 10% of requests happen during peak hours (4 hours), this results in:
  - Total Requests during Peak: 10 million requests / 24 hours * 4 hours = ~1.67 million requests.
  - QPS during Peak: 1.67 million / 14,400 seconds = ~116 QPS.
- **Storage Needs**: Assuming each task takes up 1 KB, for 1 million tasks, we need about 1 GB of storage. Considering historical data, we might estimate around 100 GB for 100 million tasks over time.

The architecture consists of the following components:
- **API Gateway**: Using **AWS API Gateway** for endpoint management and request routing.
- **Load Balancer**: Implement an **AWS Elastic Load Balancer** to distribute the incoming requests across multiple service instances.
- **Task Scheduling Service**: A microservice built using **Node.js** that handles the logic for task creation, updates, and scheduling.
- **Database**: Use **Amazon DynamoDB** for low-latency task storage, leveraging its auto-scaling capabilities to handle varying loads.
- **Caching Layer**: Implement **Redis** as an in-memory data store to cache frequently accessed task data and reduce read latency.
- **Notification Service**: A separate microservice using **AWS SNS** to handle notifications for task completions.
- **Monitoring**: Use **AWS CloudWatch** for monitoring system health and alerting on failures.

The proposed schema for the task data in DynamoDB:
- **Table: Tasks**
  - **Partition Key**: userId (String)
  - **Sort Key**: taskId (String)
  - **Attributes**:
    - taskName (String)
    - status (String)  // e.g., Scheduled, Completed, Failed
    - executionTime (DateTime)
    - recurrenceRule (String)  // e.g., daily, weekly
    - createdAt (DateTime)
    - updatedAt (DateTime)
- **Access Patterns**:
  - Query tasks by userId for display.
  - Query tasks by status for notifications.
  - Efficient sorting by execution time using the sort key.

- **Consistency vs. Availability**: Opting for eventual consistency with DynamoDB to ensure high availability, as task scheduling can tolerate slight delays in data visibility.
- **Database Choice**: Choosing DynamoDB for its scalability and low latency, accepting the trade-off of complex querying capabilities compared to relational databases.
- **Caching Strategy**: Caching frequently accessed tasks in Redis to reduce database load, accepting the trade-off of potentially serving stale data in case of a cache miss.
- **Failure Handling**: Implementing retries and dead-letter queues for failed task executions to ensure reliability, at the cost of increased complexity in error handling.

Build a low-latency Task Scheduling Pipeline

Salesforce

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Capacity Estimation

Submit Your Answer

Salesforce Product Manager Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search