Requirements

Functional Requirements:

• Schedule task at specified time
• Make recurring tasks
• Execute tasks and record success/failure
• Cancel pending task
• Pausing recurring task
• Edit task

Non-Functional Requirements:

• Low latency (thousands of tasks with minimal delay)
• High availability/reliability

Assuming no accounts (or authorization done by another system), same customer can submit multiple requests.

Capacity Estimation

Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...

API Design

• PUT task, time, repeat
• GET status of the task when asked or a notification.
• POST /tasks/{id}/cancel (or pause)
• POST /tasks/{id}/edit?task_info={new_info}

High-Level Design

Client send task to schedule which gets saved in the Postgres Database wit "Pending" status. Then we have another scheduler service that scans the Database for tasks that would be ready in the near future. Takes them to Redis time-indexed queue and with a promoter checking that queue, it can remove them to ready list in redis. The dispatcher service will check Redis and pick tasks that are ready, go to the DB to atomically update the status to "Running" and dispatch workers to execute the tasks. The workers will return with SUCCESS/FAILURE status to write in the database.

We're using Redis to lower our latency. If Redis goes down, dispatchers fall back to polling the database directly — higher latency, but no task loss since Postgres holds every task

As more tasks and to ensure high availability we can replicate our services (API, scheduler, and dispatcher). We can also scale the database horizontally and replicate and shard the data as well based on their timestamp if needed.

To absorb burst scenarios like many recurring tasks due at the same instant, the scheduler pre-loads the near-future window into Redis ahead of time so the batch is already warm, and dispatchers/workers scale horizontally to handle the spike.

For recurring tasks, we have in the database repeat counter that we can decrease and in this case we'll have to replicate that data for the task and differentiate based on timestamp+task_id. recurring tasks store a schedule; each firing generates a new PENDING instance.

Database Design

We'll have a SQL database to store one table tasks and its status

TASK (schedule_type, status, end_at, time, task_id) and index by time+task_id. I need both because task might be recurring so I want to be able to get task status for one of its runs.

We'll have a SQL database to store one table tasksand its status:

TASK (id, schedule_type, status, run_at, end_at, task_id, payload, repeat_interval, attempts)

indexed by (run_at, task_id) — I need both because a task might be recurring, so I want to get the status of any individual run.

payload — the actual work to execute (e.g., {"user_id": 123, "template": "welcome"}for an email task). The worker reads this to know what to do. Stored as JSONB for schema flexibility.

repeat_interval — the recurrence rule in seconds; the next run computes next_run_at = run_at + repeat_interval. For cron-style schedules, this becomes a cron_expression string instead. Both are NULLfor one-time tasks, which is how the system distinguishes the two.

Detailed Component Design

Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.