Celery Task Queue
Asynchronous Tasks
Python
Task Completion Callbacks
Distributed Task Processing

celery - call function on task done

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

In Celery, "run a function when the task is done" can mean different things: a second task should run after the first one, task-local cleanup should happen on success, or global post-task logging should fire for every task. Celery supports all three, but the right mechanism depends on whether you are modeling workflow, lifecycle behavior, or cross-cutting instrumentation.

If One Task Should Trigger Another, Use Canvas

If the real requirement is "when task A finishes successfully, run task B," the cleanest solution is usually a Celery canvas primitive such as chain or link.

Example with chain:

python
1from celery import Celery
2
3app = Celery(
4    "demo",
5    broker="redis://localhost:6379/0",
6    backend="redis://localhost:6379/0",
7)
8
9@app.task
10def add(x, y):
11    return x + y
12
13@app.task
14def report(result):
15    print(f"Task finished with result: {result}")
16
17if __name__ == "__main__":
18    workflow = add.s(2, 3) | report.s()
19    workflow.delay()

Here, report is not a local Python callback. It is another Celery task, which is usually what you want in a distributed task system.

If the Behavior Belongs to the Task, Override Hooks

Sometimes the follow-up behavior belongs to the task class itself, not to a larger workflow. In that case, task lifecycle hooks such as on_success or on_failure are appropriate:

python
1from celery import Celery, Task
2
3app = Celery(
4    "demo",
5    broker="redis://localhost:6379/0",
6    backend="redis://localhost:6379/0",
7)
8
9class LoggingTask(Task):
10    def on_success(self, retval, task_id, args, kwargs):
11        print(f"Task {task_id} succeeded with {retval}")
12
13@app.task(base=LoggingTask)
14def multiply(x, y):
15    return x * y

This is useful for task-specific audit messages, timing, or cleanup that should happen whenever that task succeeds.

If You Need Global Post-Task Behavior, Use Signals

Signals are a better fit when many tasks need the same after-run logic:

python
1from celery import Celery
2from celery.signals import task_postrun
3
4app = Celery(
5    "demo",
6    broker="redis://localhost:6379/0",
7    backend="redis://localhost:6379/0",
8)
9
10@task_postrun.connect
11def after_task(sender=None, task_id=None, state=None, **kwargs):
12    print(f"Task {task_id} finished with state {state}")

This is useful for:

  • metrics
  • centralized logging
  • tracing
  • generic cleanup work

Because signals are global, they should be kept lightweight and predictable.

Pick the Mechanism Based on Intent

A good rule is:

  • use chain, link, group, or chord for workflow composition
  • use task hooks for behavior owned by a task class
  • use signals for system-wide concerns

That distinction matters because Celery runs in distributed worker processes. A plain in-process callback mindset usually does not map well to task orchestration.

Example with an Error Callback

Celery can also route failures into a follow-up task:

python
1from celery import Celery
2
3app = Celery(
4    "demo",
5    broker="redis://localhost:6379/0",
6    backend="redis://localhost:6379/0",
7)
8
9@app.task
10def divide(x, y):
11    return x / y
12
13@app.task
14def handle_error(uuid):
15    print(f"Task failed: {uuid}")
16
17if __name__ == "__main__":
18    divide.apply_async(args=(10, 0), link_error=handle_error.s())

This is often better than catching everything in global hooks when the failure path is part of one explicit workflow.

Result Backends and Completion State

If you need to inspect task results after completion, you need a result backend. Without one, Celery can still execute tasks, but result retrieval is limited.

Example:

python
result = multiply.delay(6, 7)
print(result.get(timeout=10))

Be careful with get(). Blocking on task results in web requests or in other workers can defeat the purpose of asynchronous processing and can create bad dependency chains.

Common Pitfalls

  • Using a global signal when the real need is an explicit task-to-task workflow hides business logic.
  • Writing slow or failure-prone application logic inside signals makes debugging harder.
  • Calling get() everywhere turns asynchronous work back into synchronous waiting.
  • Treating Celery like local callback code ignores the fact that tasks run in distributed worker processes.
  • Forgetting to configure a result backend leads to confusion when trying to inspect completion state or return values.

Summary

  • Use canvas primitives such as chain or link when one task should run after another.
  • Override task hooks like on_success when the behavior belongs to a specific task class.
  • Use signals such as task_postrun for global logging, metrics, or cleanup.
  • Configure a result backend if you need stored return values or completion state.
  • Choose the mechanism based on whether you are modeling workflow, task lifecycle, or system-wide behavior.

Course illustration
Course illustration

All Rights Reserved.