celery - call function on task done
Master System Design with Codemia
Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.
Introduction
In Celery, "run a function when the task is done" can mean different things: a second task should run after the first one, task-local cleanup should happen on success, or global post-task logging should fire for every task. Celery supports all three, but the right mechanism depends on whether you are modeling workflow, lifecycle behavior, or cross-cutting instrumentation.
If One Task Should Trigger Another, Use Canvas
If the real requirement is "when task A finishes successfully, run task B," the cleanest solution is usually a Celery canvas primitive such as chain or link.
Example with chain:
Here, report is not a local Python callback. It is another Celery task, which is usually what you want in a distributed task system.
If the Behavior Belongs to the Task, Override Hooks
Sometimes the follow-up behavior belongs to the task class itself, not to a larger workflow. In that case, task lifecycle hooks such as on_success or on_failure are appropriate:
This is useful for task-specific audit messages, timing, or cleanup that should happen whenever that task succeeds.
If You Need Global Post-Task Behavior, Use Signals
Signals are a better fit when many tasks need the same after-run logic:
This is useful for:
- metrics
- centralized logging
- tracing
- generic cleanup work
Because signals are global, they should be kept lightweight and predictable.
Pick the Mechanism Based on Intent
A good rule is:
- use
chain,link,group, orchordfor workflow composition - use task hooks for behavior owned by a task class
- use signals for system-wide concerns
That distinction matters because Celery runs in distributed worker processes. A plain in-process callback mindset usually does not map well to task orchestration.
Example with an Error Callback
Celery can also route failures into a follow-up task:
This is often better than catching everything in global hooks when the failure path is part of one explicit workflow.
Result Backends and Completion State
If you need to inspect task results after completion, you need a result backend. Without one, Celery can still execute tasks, but result retrieval is limited.
Example:
Be careful with get(). Blocking on task results in web requests or in other workers can defeat the purpose of asynchronous processing and can create bad dependency chains.
Common Pitfalls
- Using a global signal when the real need is an explicit task-to-task workflow hides business logic.
- Writing slow or failure-prone application logic inside signals makes debugging harder.
- Calling
get()everywhere turns asynchronous work back into synchronous waiting. - Treating Celery like local callback code ignores the fact that tasks run in distributed worker processes.
- Forgetting to configure a result backend leads to confusion when trying to inspect completion state or return values.
Summary
- Use canvas primitives such as
chainorlinkwhen one task should run after another. - Override task hooks like
on_successwhen the behavior belongs to a specific task class. - Use signals such as
task_postrunfor global logging, metrics, or cleanup. - Configure a result backend if you need stored return values or completion state.
- Choose the mechanism based on whether you are modeling workflow, task lifecycle, or system-wide behavior.

