Design a Task Scheduling Service

Question

Design a fault-tolerant task scheduling system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

### Functional Requirements:
1. **User Authentication**: Users must be able to authenticate via OAuth or API tokens.
2. **Task Creation**: Users can create, update, and delete scheduled tasks.
3. **Task Execution**: The system must execute tasks at specified times, including support for recurring tasks.
4. **Task Status Tracking**: Users must be able to query the status of tasks (e.g., pending, in progress, completed, failed).
5. **Error Handling**: The system must log errors and provide notifications for task failures.

### Non-Functional Requirements:
1. **Scalability**: The service should handle millions of requests with low latency.
2. **Fault Tolerance**: The system must be resilient to failures, ensuring tasks are not lost.
3. **High Availability**: The service should have an uptime of 99.9%.
4. **Security**: Data in transit and at rest must be encrypted, and access must be restricted based on user roles.

Assuming Neon expects to handle 10 million tasks per day:
- **Tasks per second**: 10 million tasks / 86400 seconds = ~115.7 tasks/second.
- **Peak Load**: During peak hours, assume 10% of daily tasks are created. This results in 1,157 tasks/second.
- **Storage Requirements**: Each task may require 1KB of storage. For 10 million tasks, that’s approximately 10 GB of storage per day. Over a month, this totals 300 GB.
- **Assumptions**: Each task will have metadata (e.g., user ID, status, timestamps), leading to a total estimate of ~1TB per year.

### Component Diagram:
- **API Gateway**: Handles incoming requests and routes them to the appropriate services.
- **Task Service**: Core service responsible for managing task lifecycle (creation, update, deletion).
- **Scheduler**: A cron-like service that checks for tasks to execute based on their schedule.
- **Database**: A distributed database (e.g., MongoDB) for storing task data.
- **Cache**: Redis for caching task statuses to reduce database load.
- **Notification Service**: Sends alerts for task failures or completions.

### Technology Choices:
- **API Gateway**: Kong or AWS API Gateway.
- **Task Service**: Node.js for handling asynchronous requests.
- **Database**: MongoDB for flexible schema and scaling.
- **Cache**: Redis for fast access to frequently queried task statuses.

### Schema Design:
- **Tasks Collection**:
  - `task_id`: String (unique identifier)
  - `user_id`: String (identifier for the user)
  - `status`: Enum (pending, in_progress, completed, failed)
  - `schedule_time`: Date (when the task should execute)
  - `recurrence`: Object (optional, contains recurrence rules)
  - `created_at`: Date (timestamp for task creation)
  - `updated_at`: Date (timestamp for last update)

### Access Patterns:
- **Queries by User**: Fetch tasks based on `user_id`.
- **Status Updates**: Update status based on `task_id`.
- **Scheduled Execution**: Retrieve tasks with `schedule_time` <= current time.

### Architectural Decisions:
1. **Consistency vs. Availability**: Opting for eventual consistency to maximize availability. Users can see task status updates with slight delays, which is acceptable in most use cases.
2. **Database Choice**: Choosing MongoDB for flexible schema design and horizontal scaling capabilities, but sacrificing some immediate consistency guarantees in favor of availability and partition tolerance.
3. **Caching Strategy**: Using Redis introduces complexity but significantly improves read performance for frequently accessed task statuses, reducing database load.
4. **Task Scheduler**: A trade-off in ensuring tasks are executed on time versus the overhead of managing many scheduled tasks. Using a distributed cron-like service balances execution reliability with resource efficiency.

Design a Task Scheduling Service

Neon

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Functional Requirements:

Capacity Estimation

Submit Your Answer

Neon Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search