FastAPI
background tasks
troubleshooting
Python
asynchronous programming

Background Task in FastAPI is not working properly

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

FastAPI BackgroundTasks works well for small follow-up jobs such as logging, email notifications, or lightweight file writes. When it "doesn't work properly," the real issue is often that the task is too heavy, exceptions are going unnoticed, or the developer expects queue-like durability from a feature that runs in the same application process.

What BackgroundTasks Actually Does

BackgroundTasks schedules a callable to run after the response is prepared. It is convenient, but it is not a distributed worker system and it is not meant for long, CPU-heavy jobs.

python
1from fastapi import BackgroundTasks, FastAPI
2
3app = FastAPI()
4
5
6def write_log(message: str) -> None:
7    with open("app.log", "a", encoding="utf-8") as handle:
8        handle.write(message + "\n")
9
10
11@app.post("/notify")
12async def notify(background_tasks: BackgroundTasks):
13    background_tasks.add_task(write_log, "request received")
14    return {"status": "queued"}

That is the intended scale of usage: short post-response work inside the same service process.

Why Tasks Seem to Fail

A few issues show up repeatedly:

  • the task function throws an exception and nothing logs it clearly
  • the task is CPU-bound and blocks the worker
  • the process restarts before the task finishes
  • the developer expects retries or durable delivery

BackgroundTasks is best understood as "run this later in the app lifecycle," not "guarantee this work happens no matter what."

Make the Task Independently Correct

Before debugging FastAPI itself, run the task function directly. If it fails outside the web request, it will fail inside the background task too.

Also add explicit error handling:

python
1def write_log_safe(message: str) -> None:
2    try:
3        with open("app.log", "a", encoding="utf-8") as handle:
4            handle.write(message + "\n")
5    except Exception as exc:
6        print(f"background task failed: {exc}")

This is especially important because background-task failures can be less obvious than request-handler failures.

Know When to Use a Real Task Queue

If the work is slow, CPU-heavy, or must survive process restarts, BackgroundTasks is the wrong tool. In those cases, use a proper queue system such as Celery, RQ, or another worker architecture.

That is the dividing line:

  • 'BackgroundTasks for short in-process follow-up work'
  • task queue for durable, retriable, or heavy jobs

Trying to stretch BackgroundTasks into a job system usually leads to the "it works sometimes" class of bugs.

Async Versus Sync Functions

FastAPI can schedule both sync and async callables, but that does not eliminate resource constraints. An async def task is still a poor fit for heavy CPU work, and a sync task can still block meaningful progress if it does too much.

So the right question is not only "sync or async?" It is also "how much work am I trying to do inside the API worker process?"

Common Pitfalls

One common mistake is assuming BackgroundTasks behaves like a durable queue. It does not retry failed work and does not survive every shutdown scenario.

Another issue is hiding task errors by not logging exceptions inside the background function.

It is also easy to use it for expensive work just because the response returns quickly. A fast HTTP response does not mean the server has the right execution model for the background job.

Summary

  • FastAPI BackgroundTasks is meant for short in-process work after the response is created.
  • It is not a durable task queue and should not be treated like one.
  • Debug the task function independently and log exceptions inside it.
  • Use a real worker system for heavy, slow, or must-not-lose jobs.
  • A quick HTTP response does not guarantee the background task model fits the workload.

Course illustration
Course illustration

All Rights Reserved.