NestJS - Task Scheduling - Prevent running the same Job in parallel in identical service instances in K8s
Master System Design with Codemia
Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.
Introduction
If multiple identical NestJS Pods run the same scheduled task in Kubernetes, every instance will trigger the job unless you coordinate them explicitly. A local in-memory flag is not enough because each Pod has its own memory space. To prevent parallel execution, you need a distributed lock or you need to move the scheduling responsibility out of the application layer entirely.
Why the Problem Happens
@nestjs/schedule runs inside each application process. If you deploy three replicas, each one evaluates the cron expression independently.
Simple scheduled job:
With multiple Pods, that code runs once per Pod. Kubernetes does not automatically deduplicate in-process cron jobs.
Do Not Rely on In-Memory Guards
This does not solve the problem in Kubernetes:
It only prevents overlap inside one process. Another Pod knows nothing about that flag.
If the job must be singleton across replicas, the lock must live in shared infrastructure such as Redis, Postgres, or another coordination system.
Redis Lock Example
A common approach is a short-lived distributed lock in Redis.
The NX option means “only set if absent.” That turns Redis into a simple lock coordinator.
Use a Lock Expiration
Every distributed lock needs a timeout or lease. If a Pod crashes while holding the lock, the lock must eventually expire or the job will stop forever.
That is why the Redis example uses a TTL. Set the TTL long enough to cover normal execution time, but not so long that crash recovery becomes slow.
If the job duration can exceed the TTL, implement lock renewal or use a more robust coordination approach.
Database Lock Alternative
If you already depend on Postgres, an advisory lock can be a good fit.
That avoids introducing Redis only for locking. The tradeoff is tying the scheduling coordination to database availability and transaction behavior.
Sometimes the Right Answer Is Kubernetes CronJob
If the task is truly scheduled infrastructure work and does not need to live inside the web service process, Kubernetes CronJob is often cleaner than NestJS in-process scheduling.
Benefits:
- single scheduling authority
- no distributed lock logic in app code
- retries and history handled by Kubernetes
NestJS scheduling is convenient, but not always the best abstraction for singleton cluster-wide jobs.
Make the Job Idempotent Anyway
Even with locks, jobs should be written so duplicate execution is not catastrophic. Locks reduce the probability of overlap; idempotency reduces the damage if overlap still happens because of clock drift, TTL expiry, or split-brain style issues.
Examples of idempotency:
- upsert instead of blind insert
- mark processed rows with stable identifiers
- skip already-completed work
This is distributed-systems hygiene, not optional polish.
Common Pitfalls
The biggest mistake is using an in-memory boolean and assuming it prevents overlap across Pods. It does not.
Another issue is acquiring a distributed lock without a TTL. If the worker crashes, the lock can become permanent.
Teams also sometimes choose app-level scheduling when Kubernetes CronJob would be operationally simpler and clearer.
Summary
- NestJS scheduled jobs run in every replica unless you coordinate them.
- In-memory guards only work inside one process, not across Pods.
- Use a distributed lock such as Redis or database advisory locking for singleton execution.
- Always give the lock a TTL or lease strategy.
- Prefer Kubernetes CronJob when the job is cluster-level infrastructure work rather than app-local behavior.

