Build a geo-distributed Task Scheduling Pipeline

Question

Design a geo-distributed task scheduling system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

### Functional Requirements:
1. **Task Scheduling**: Ability to schedule and execute tasks across multiple geographic regions.
2. **Geo-Distribution**: Support for region-specific task execution, minimizing latency for users in different locations.
3. **Scalability**: Handle millions of requests per day, scaling horizontally to accommodate varying loads.
4. **Monitoring**: Real-time monitoring of task status, execution times, and failure rates.
5. **Feedback Loop**: Ability to collect performance metrics from executed tasks for model retraining.

### Non-Functional Requirements:
1. **Availability**: 99.9% uptime to ensure continuous task scheduling.
2. **Consistency**: Eventual consistency for task states, allowing for delay in updates across regions.
3. **Latency**: Average task execution response time of less than 100ms.
4. **Security**: Ensure data integrity and confidentiality through encryption and access controls.

Assuming JPMorgan handles about 10 million requests per day, we can break this down into:
- **Peak Traffic**: Let's say 20% of requests occur during peak hours (2 million requests).
- **Requests Per Second (RPS)**:

RPS = 2,000,000 requests / (4 hours * 3600 seconds) = ~138.9 RPS.
  
Assuming a 3x replication factor for fault tolerance, we would expect to handle approximately 417 RPS across three data centers.

### Storage Needs:
- **Task Data**: Each task might require ~1KB of storage. 
- **Daily Storage = 10 million tasks * 1KB = ~10 GB/day.**
- **Monthly Storage = 300 GB/month.**
- Using a database like Cassandra, we would scale our storage to handle ~3TB/year.

### High-Level Architecture Diagram:
- **Components**:
  1. **API Gateway**: Nginx or AWS API Gateway to route requests.
  2. **Task Scheduler**: Microservice using Spring Boot or Node.js to manage task scheduling logic.
  3. **Task Queue**: RabbitMQ or Kafka for handling task queuing and processing.
  4. **Worker Nodes**: Docker containers running tasks, orchestrated by Kubernetes across data centers.
  5. **Monitoring Service**: Prometheus and Grafana for real-time metrics and alerting.
  6. **Database**: Cassandra for storing task metadata, providing high availability and horizontal scalability.

### Data Flow:
1. Users submit tasks via the API Gateway.
2. API Gateway routes to Task Scheduler.
3. Task Scheduler places tasks into the Task Queue.
4. Worker Nodes pull tasks from the queue and execute them.
5. Task status updates are sent back to the database.

### Schema Design:
**Task Table (Cassandra)**:
- `task_id` (UUID, PRIMARY KEY)
- `user_id` (UUID)
- `task_type` (TEXT)
- `task_data` (TEXT)
- `status` (TEXT) - (Pending, In-Progress, Completed, Failed)
- `created_at` (TIMESTAMP)
- `updated_at` (TIMESTAMP)

### Access Patterns:
- **Task Lookup**: By `task_id` for tracking individual task status.
- **User Tasks**: Query by `user_id` to retrieve all tasks for a given user.
- **Status Aggregation**: Query by `status` to monitor system health and task distribution.

### Architectural Decisions:
1. **Eventual Consistency vs. Strong Consistency**: Opting for eventual consistency allows for better availability and partition tolerance across geo-distributed systems. However, this means users might see stale data for a short period.

2. **Complexity vs. Latency**: Using microservices adds complexity but allows for independent scaling of components. For tasks requiring low latency, we may consider caching frequently accessed data using Redis to improve response times.

3. **Batch Processing vs. Real-time Processing**: While real-time processing is beneficial for immediate task execution feedback, it may introduce higher complexity in managing task failures. Batch processing could be employed for non-time-sensitive tasks to reduce overhead.

4. **Monitoring Trade-offs**: Implementing a comprehensive monitoring system adds overhead and operational complexity but is crucial for ensuring system reliability and for enabling effective feedback loops for model retraining.

Build a geo-distributed Task Scheduling Pipeline

JPMorgan

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Functional Requirements:

Capacity Estimation

Submit Your Answer

JPMorgan Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search