Design Task Scheduling Infrastructure for global users

Question

Design a real-time task scheduling system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

- **Functional Requirements**:
  - **Task Creation**: Users can create, update, and delete tasks in real-time.
  - **Task Scheduling**: Ability to schedule tasks for specific times and priorities.
  - **Notifications**: Notify users when tasks are created, updated, or cancelled.
  - **Rate Limiting**: Implement API rate limiting to manage request load and prevent abuse.

- **Non-Functional Requirements**:
  - **Scalability**: System must support millions of concurrent users with low latency.
  - **Availability**: 99.99% uptime, ensuring global accessibility.
  - **Consistency**: Data should be consistent across regions with eventual consistency as a fallback.
  - **Performance**: Response times under 100ms for 95% of requests.

Assuming DoorDash has 10 million active users globally, with an average of 10 tasks created per user per day:
- **Total Tasks per Day**: 10M users * 10 tasks = 100M tasks/day.
- **Tasks per Second**: 100M tasks / 86400 seconds = ~1157 tasks/second.
- **Peak Load**: Assuming peak usage of 5x average, we can estimate peak tasks at ~5785 tasks/second.
- **Storage Needs**: Assuming an average task size of 1KB, we would need about 100GB of storage per day, translating to 36.5TB per year.

The architecture consists of the following components:
- **API Gateway**: AWS API Gateway for managing incoming requests and rate limiting.
- **Task Service**: A microservice (Node.js) responsible for task operations (CRUD).
- **Database**: Amazon DynamoDB for storing tasks, chosen for its scalability and global distribution.
- **Caching Layer**: Redis for caching frequently accessed tasks to reduce database load.
- **Notification Service**: AWS SNS for sending real-time notifications to users.
- **Load Balancer**: AWS ELB to distribute incoming traffic evenly across multiple instances.

![High-Level Architecture](https://example.com/architecture-diagram.png)

The schema for the task model in DynamoDB:
- **Table Name**: Tasks
- **Primary Key**: `taskId` (String)
- **Attributes**:
  - `userId` (String)
  - `taskDescription` (String)
  - `status` (String - ENUM: PENDING, COMPLETED, CANCELLED)
  - `scheduledTime` (Timestamp)
  - `priority` (Integer)

- **Access Patterns**:
  - Query by `userId` to retrieve all tasks for a specific user.
  - Use GSI (Global Secondary Index) on `scheduledTime` for scheduling queries.

- **Consistency vs. Availability**: With a global user base, we prioritize availability, leading to eventual consistency. Conflicts can be resolved using timestamps for the latest update.
- **Caching vs. Freshness**: Using Redis introduces complexity around cache invalidation but significantly improves read performance. We will implement cache expiration and updates upon task modification.
- **Data Model Complexity**: Choosing a NoSQL database like DynamoDB provides scalability but requires careful design to manage relationships and queries effectively. We will leverage denormalization for common access patterns.

Design Task Scheduling Infrastructure for global users

DoorDash

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Capacity Estimation

Submit Your Answer

DoorDash Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search