Architect a real-time Task Scheduling Engine

Question

Design a real-time task scheduling system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

- **Functional Requirements:**  
  - Ability to handle real-time scheduling requests for tasks with a response time of under 100 milliseconds.  
  - Support for priority-based task scheduling, where higher priority tasks can preempt lower priority ones.  
  - Integration with existing Apple services for user authentication and data access.  
  - Provide an API for clients to submit tasks and monitor their status.  
  - Support task retries and failure handling, notifying clients of task completion or failure.

- **Non-Functional Requirements:**  
  - **Scalability:** Must support millions of requests per minute, particularly during peak hours like software updates.  
  - **Availability:** Aim for 99.99% uptime to ensure users can always schedule tasks.  
  - **Consistency:** Eventual consistency for task completion statuses is acceptable, but strong consistency for task submissions is required.  
  - **Performance:** Latency of under 100 ms for task scheduling requests and 200 ms for task status retrieval.

- **User Base Estimation:**  
  - Assume Apple has 1 billion active devices. If 10% of users schedule tasks during peak usage, that results in 100 million requests.  
  - Peak request rate estimation: 100 million requests/hour translates to approximately 27,800 requests/second.

- **Task Payload Size:**  
  - Average size of a task request, including metadata, is approximately 1 KB.  
  - Total data per second = 27,800 requests * 1 KB = 27.8 MB/sec.

- **Storage Needs:**  
  - Assuming tasks need to be stored for 1 week for monitoring and analytics.  
  - 27.8 MB/sec * 7 days = 2,265 GB ≈ 2.3 TB of storage per week.

- **Component Diagram:**  
  - **API Gateway:** Handles incoming requests and routes them to appropriate services.  
  - **Task Scheduler Service:** Core service that processes task submissions and manages the scheduling queue.  
  - **Database:** Use a combination of Redis for fast access to active tasks and PostgreSQL for persistent storage of task history.  
  - **Monitoring Service:** Tracks task success rates, latencies, and provides alerts for anomalies.  
  - **Feature Engineering Service:** Prepares features from user data for ML models to predict task completion times.  
  - **ML Serving Layer:** Hosts the trained models for predicting task priority and estimated completion time.  
  - **Feedback Loop:** Collects data on task performance for retraining models.

- **Technology Choices:**  
  - API Gateway: AWS API Gateway or NGINX.  
  - Task Scheduler: Node.js or Go for high concurrency.  
  - Database: Redis for in-memory caching, PostgreSQL for relational data.  
  - ML Framework: TensorFlow or PyTorch for model training.

- **Schema Design:**  
  - **Task Table:**  
    - `task_id` (Primary Key)  
    - `user_id`  
    - `task_type`  
    - `priority` (Integer)  
    - `status` (Enum: PENDING, IN_PROGRESS, COMPLETED, FAILED)  
    - `submission_time` (Timestamp)  
    - `completion_time` (Timestamp)  
    - `input_data` (JSON)

- **Access Patterns:**  
  - Frequent reads for task status updates, hence Redis for caching.  
  - Bulk reads for analytics, hence PostgreSQL for storing historical task data.

- **Consistency vs. Availability:**  
  - Chose eventual consistency for task completion statuses to improve availability.  
  - Strong consistency enforced for task submissions to ensure that users do not receive conflicting information.

- **Complexity vs. Latency:**  
  - Opted for simpler models for predicting task completion to ensure lower latency in real-time serving.  
  - This may sacrifice some accuracy but ensures responsiveness in user experience.

- **Monitoring vs. Performance:**  
  - Implemented comprehensive monitoring that may introduce slight overhead but is essential for maintaining service reliability and detecting degradation in model performance.

Architect a real-time Task Scheduling Engine

Apple

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Capacity Estimation

Submit Your Answer

Apple Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search