Introduction

Measuring Celery task execution time is useful for spotting slow jobs, tuning worker concurrency, and alerting on degraded performance. The usual approaches are application-level timing with Celery signals, or reading task runtime information from Celery events and monitoring tools.

A Simple Signal-Based Timing Pattern

Celery provides task lifecycle signals that let you record a start time and compute the duration when the task finishes.

1import time
2from celery import Celery
3from celery.signals import task_prerun, task_postrun
4
5app = Celery("tasks", broker="pyamqp://guest@localhost//")
6
7task_start_times = {}
8
9@task_prerun.connect
10def record_start(sender=None, task_id=None, **kwargs):
11    task_start_times[task_id] = time.perf_counter()
12
13@task_postrun.connect
14def record_finish(sender=None, task_id=None, task=None, **kwargs):
15    started = task_start_times.pop(task_id, None)
16    if started is not None:
17        duration = time.perf_counter() - started
18        print(f"{task.name}[{task_id}] took {duration:.4f}s")
19
20@app.task
21def add(x, y):
22    time.sleep(1.2)
23    return x + y

This is easy to add and works well for local diagnostics or small deployments.

Prefer a Monotonic Clock

Use time.perf_counter() or another monotonic timer rather than time.time(). Wall-clock time can jump if the system clock changes, while a monotonic timer is designed for measuring elapsed duration reliably.

That small choice matters more than most people expect in long-running worker processes.

Instrument the Task Directly When You Need More Context

Sometimes a global signal hook is too generic. If you need tags, business identifiers, or custom metrics, measure inside the task body itself.

1import time
2from celery import shared_task
3
4@shared_task
5def process_order(order_id):
6    started = time.perf_counter()
7    try:
8        # real work here
9        time.sleep(0.8)
10        return {"order_id": order_id, "status": "ok"}
11    finally:
12        duration = time.perf_counter() - started
13        print(f"process_order({order_id}) took {duration:.4f}s")

This pattern is useful when execution time should be reported alongside task-specific metadata.

Send Metrics to a Monitoring System

Printing durations is fine for development, but production systems usually need metrics that can be aggregated. A simple pattern is to push timings to StatsD, Prometheus, or another metrics backend.

def send_metric(task_name, duration_seconds):
    print(f"METRIC celery.task.duration task={task_name} value={duration_seconds}")

Then call that helper from the signal or task-level timing code. The important thing is consistency: use one metric name and labeling convention across tasks so dashboards remain usable.

Celery Events and External Monitoring

Celery can also emit runtime events that tools such as Flower or custom event consumers can observe. That approach is useful when you want cluster-wide monitoring without embedding too much logic in every task module.

A monitoring process can subscribe to events and read task state transitions, including success and runtime data when available. That is often a better fit for operational dashboards than ad hoc log prints.

Queue Time Versus Execution Time

Be clear about what you are measuring. There is a big difference between:

• time spent waiting in the queue
• time spent executing on the worker

Signal-based timing around task execution measures worker runtime, not queue delay. If users care about end-to-end latency, you may also need to record the time between task publication and task start.

Common Pitfalls

The biggest mistake is using wall-clock time instead of a monotonic timer. Clock adjustments can make short timing measurements misleading.

Another issue is storing task start times globally without cleanup. If the dictionary grows forever because of exceptions or missing cleanup, the monitoring code becomes its own problem.

Developers also confuse runtime with end-to-end latency. A task that executes quickly can still feel slow if it sits in the queue for a long time.

Finally, console logging is not enough for production observability. If execution time matters operationally, emit structured metrics or events that your monitoring stack can aggregate.

Summary

• Use task_prerun and task_postrun signals for simple execution-time measurement.
• Prefer time.perf_counter() for reliable elapsed-time tracking.
• Instrument tasks directly when you need richer task-specific timing context.
• Send metrics to a monitoring system instead of relying on ad hoc prints in production.
• Distinguish worker execution time from total queue-to-result latency.