Celery
asynchronous tasks
task chains
positional arguments
Python

Groups of chains with positional arguments in partial tasks using Celery

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

Celery workflows become confusing when chain, group, and partial task signatures are combined without a clear mental model for argument passing. The most important rule is that chained tasks receive the previous task’s result as the first positional argument unless you intentionally make the next signature immutable.

How Positional Arguments Flow in chain

In a Celery chain, the result of one task is injected into the next task automatically.

python
1from celery import Celery, chain
2
3app = Celery(
4    "demo",
5    broker="redis://localhost:6379/0",
6    backend="redis://localhost:6379/0",
7)
8
9
10@app.task
11def add(a, b):
12    return a + b
13
14
15@app.task
16def multiply(value, factor):
17    return value * factor
18
19
20workflow = chain(add.s(2, 3), multiply.s(10))
21print(workflow.apply_async().get(timeout=10))

multiply.s(10) does not mean “call multiply with only one argument.” It means “when the previous result arrives, insert it first, then append 10.”

So the actual call becomes:

python
multiply(5, 10)

That is the key behavior behind most Celery positional-argument surprises.

Grouping Several Chains

If you need multiple independent chains running in parallel, wrap them in a group.

python
1from celery import group, chain
2
3jobs = group(
4    chain(add.s(1, 2), multiply.s(3)),
5    chain(add.s(4, 5), multiply.s(6)),
6    chain(add.s(7, 8), multiply.s(9)),
7)
8
9result = jobs.apply_async().get(timeout=20)
10print(result)

Each chain handles its own positional argument flow internally, and the group returns the list of chain results.

The important point is that group does not rewrite the argument semantics of chain. It only runs several signatures in parallel.

What “Partial” Means in Celery Signatures

When you call .s(...), you are creating a partial signature. It pre-fills some arguments now and leaves room for Celery to inject previous results later.

That is why .s(...) is so useful in chains. It lets you say:

  • take whatever the previous task returns
  • then call the next task with these extra positional arguments too

But that convenience can also become a bug if the downstream task was not designed to accept the injected first argument.

Use .si(...) to Stop Result Injection

If a task should ignore the previous result and use only its own explicit arguments, use an immutable signature with .si(...).

python
1@app.task
2def log_message(level, text):
3    return f"{level}: {text}"
4
5
6workflow = chain(add.s(2, 2), log_message.si("INFO", "done"))
7print(workflow.apply_async().get(timeout=10))

Without .si(...), Celery would try to pass the result of add into log_message as the first argument, which would shift the intended positions and usually break the call.

This is the most important fix when chained workflows keep producing “wrong number of arguments” or “unexpected value in first parameter” errors.

Prefer Named Parameters in Complex Workflows

Deep positional pipelines are fragile. If a task signature changes, every downstream assumption about argument order may break.

When practical, use keyword arguments:

python
1@app.task
2def score_event(user_id, points, source=None):
3    return {"user_id": user_id, "points": points, "source": source}
4
5
6sig = score_event.s(user_id=42, points=10, source="checkout")
7print(sig.apply_async().get(timeout=10))

This does not remove all Celery workflow complexity, but it makes the task contract more obvious and future refactors safer.

A Chord-Style Aggregation Example

A common pattern is several chains in parallel followed by one reducer:

python
1from celery import chord, group, chain
2
3
4@app.task
5def collect_sum(values):
6    return sum(values)
7
8
9header = group(
10    chain(add.s(1, 2), multiply.s(2)),
11    chain(add.s(3, 4), multiply.s(2)),
12    chain(add.s(5, 6), multiply.s(2)),
13)
14
15workflow = chord(header)(collect_sum.s())
16print(workflow.get(timeout=20))

Here the callback receives the group result list as its first positional argument. That is consistent with the same argument-injection model, just at the group result level instead of the single-task level.

Debugging Signature Wiring

If argument flow becomes hard to reason about, add a temporary debug task:

python
1@app.task
2def debug_args(*args):
3    print("debug args:", args)
4    return args
5
6
7workflow = chain(add.s(3, 4), debug_args.s("marker"))
8print(workflow.apply_async().get(timeout=10))

This is often faster than mentally simulating a large nested signature tree.

Common Pitfalls

The most common pitfall is forgetting that a chained task receives the previous result as the first positional argument.

Another mistake is using .s(...) when .si(...) is required. That causes unplanned result injection and broken argument order.

A third issue is designing tasks with too many positional parameters, which makes workflow wiring brittle and hard to read.

Finally, developers often test only isolated tasks instead of the composed workflow. Signature problems usually appear in composition, not in single-task unit tests.

Summary

  • In a Celery chain, the previous task result is injected into the next task as the first positional argument.
  • '.s(...) creates a partial signature that still accepts that injected result.'
  • '.si(...) creates an immutable signature that ignores previous-result injection.'
  • 'group runs multiple chains in parallel but does not change chain argument semantics.'
  • Keep task contracts explicit and debug signature wiring early when composing larger workflows.

Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

All Rights Reserved.