Design Task Scheduling Infrastructure for IoT devices

Question

Design a scalable task scheduling system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

- **Functional Requirements**:
  1. Ability to schedule, execute, and monitor tasks for millions of IoT devices.
  2. Support for different types of tasks (e.g., data collection, device configuration).
  3. Provide an API for devices to submit task requests and receive status updates.
  4. Implement retry logic for failed tasks with exponential backoff.
  5. Support dynamic scaling based on load and task complexity.

- **Non-Functional Requirements**:
  1. **Scalability**: Must handle up to 10 million task requests per minute.
  2. **Availability**: 99.9% uptime, with minimal downtime during maintenance.
  3. **Latency**: Task scheduling and acknowledgment within 100 milliseconds.
  4. **Consistency**: Eventual consistency for task state across distributed nodes.
  5. **Security**: Ensure secure communication between devices and the scheduler.

- **Request Per Second (QPS)**: Assuming each IoT device sends 1 task per minute, with 10 million devices, we have:
  
  $ \text{Total QPS} = \frac{10,000,000 \text{ tasks}}{60 \text{ seconds}} = 166,667 \text{ requests/second} $
  
- **Storage Requirements**: Assuming each task requires 1KB of metadata and we retain task logs for 30 days:
  
  $ \text{Total storage} = 166,667 \text{ tasks/second} \times 30 \text{ days} \times 24 \text{ hours} \times 60 \text{ minutes} \times 60 \text{ seconds} \times 1 \text{ KB} \approx 13.7 TB $
  
- **Bandwidth**: Assuming each task response is about 500 bytes:
  
  $ \text{Total bandwidth} = 166,667 \text{ requests/second} \times 0.5 \text{ KB} \approx 83.3 MB/s $

- **Architecture Components**:
  1. **API Gateway**: For incoming task requests, using AWS API Gateway for rate limiting and security.
  2. **Task Scheduler**: Microservice for scheduling tasks, implemented using a scalable framework like Kubernetes.
  3. **Task Queue**: Using Apache Kafka for handling task requests and ensuring message durability.
  4. **Worker Nodes**: Auto-scaling groups of worker nodes (e.g., using Databricks clusters) to process tasks.
  5. **Database**: Use a distributed NoSQL database like Cassandra for storing task metadata and states.
  6. **Monitoring Service**: Use Prometheus and Grafana for monitoring task execution and system health.

- **Component Diagram**: 
  ![High-Level Architecture](https://example.com/component-diagram.png)

- **Schema Design**:
  - **Task Table**:
    - **task_id** (UUID, Primary Key)
    - **device_id** (String, Indexed)
    - **task_type** (String)
    - **status** (String, e.g., pending, in_progress, completed, failed)
    - **created_at** (Timestamp)
    - **updated_at** (Timestamp)
    - **retry_count** (Integer)
    - **payload** (Blob)

- **Access Patterns**:
  1. Query by `device_id` to fetch pending tasks.
  2. Update `status` and `retry_count` during task processing.
  3. Aggregate tasks by `status` for monitoring purposes.

- **Consistency vs. Availability**: Given the high volume of requests, we prioritize availability with eventual consistency. This ensures that the system remains responsive while allowing for some delay in task state updates.
  
- **Database Selection**: Choosing Cassandra for its write-heavy capabilities and horizontal scalability, but it comes with challenges around complex queries and joins.
  
- **Task Retry Logic**: Implementing exponential backoff for retries increases latency for failing tasks, but improves overall system stability and prevents overload.
  
- **Load Balancing**: Using a round-robin strategy for distributing tasks to worker nodes can lead to uneven load distribution, but it simplifies implementation compared to more complex algorithms.
  
- **Failure Handling**: While using Kafka for message durability ensures task persistence, it introduces additional latency in task acknowledgment.

Design Task Scheduling Infrastructure for IoT devices

Databricks

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Capacity Estimation

Submit Your Answer

Databricks Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search