Flask
Asynchronous
Python
Task Management
Web Development

Making an asynchronous task in Flask

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

In Flask, a truly asynchronous background task usually means moving work out of the request-response cycle and into a worker process. For simple demos you can start a thread, but for production-style jobs such as sending email, generating reports, or calling slow external services, a task queue such as Celery is the more reliable pattern.

Why a Background Worker Is Better Than Blocking the Request

If a Flask route performs a slow job directly, the web worker is tied up until the job finishes. That hurts throughput and makes the client wait longer than necessary.

A better flow is:

  1. receive the request
  2. enqueue the work
  3. return a task id immediately
  4. let a worker process the job in the background
  5. provide a status endpoint if the client needs progress or results

This design keeps the HTTP layer responsive and separates web concerns from job execution.

Minimal Flask and Celery Setup

A common starter stack is Flask plus Celery with Redis as both broker and result backend.

bash
python -m pip install flask celery redis

Create an app.py file like this:

python
1import time
2from flask import Flask, jsonify, request
3from celery import Celery
4
5app = Flask(__name__)
6
7celery = Celery(
8    app.import_name,
9    broker="redis://localhost:6379/0",
10    backend="redis://localhost:6379/0",
11)
12
13@celery.task
14def long_task(name):
15    time.sleep(5)
16    return {"message": f"finished task for {name}"}
17
18@app.post("/tasks")
19def start_task():
20    payload = request.get_json(force=True)
21    name = payload.get("name", "anonymous")
22    task = long_task.delay(name)
23    return jsonify({"task_id": task.id}), 202
24
25@app.get("/tasks/<task_id>")
26def task_status(task_id):
27    result = celery.AsyncResult(task_id)
28
29    if result.state == "PENDING":
30        return jsonify({"state": result.state}), 202
31    if result.state == "FAILURE":
32        return jsonify({"state": result.state, "error": str(result.info)}), 500
33    if result.state == "SUCCESS":
34        return jsonify({"state": result.state, "result": result.result})
35
36    return jsonify({"state": result.state}), 202

This example does three important things:

  • defines a background task
  • enqueues it with .delay()
  • exposes a status endpoint using the task id

Run the Web App and the Worker Separately

Celery works because the worker process is separate from Flask. You need both processes running.

Start Redis first, then run:

bash
flask --app app run

In another terminal, start the worker:

bash
celery -A app.celery worker --loglevel=info

Now send a request:

bash
curl -X POST http://127.0.0.1:5000/tasks \
  -H 'Content-Type: application/json' \
  -d '{"name": "report"}'

The response should contain a task id. Query the status endpoint with that id until the task reaches SUCCESS.

Know When Simpler Approaches Are Acceptable

For very small local tools, some developers use a background thread:

python
1import threading
2import time
3from flask import Flask
4
5app = Flask(__name__)
6
7def do_work():
8    time.sleep(3)
9    print("background work finished")
10
11@app.get("/run")
12def run_task():
13    threading.Thread(target=do_work, daemon=True).start()
14    return "started", 202

This can be fine for lightweight, non-critical tasks in a single-process development setup. It is not a substitute for a proper queue when you need retries, persistence, monitoring, or multiple web workers. If the process restarts, the thread and its work are simply gone.

Design the API Around Asynchronous Behavior

Once work moves to the background, the HTTP contract changes. A task endpoint should normally return 202 Accepted instead of pretending the work is already done. Clients should then poll a status route or receive a callback from some other system.

That design choice matters because it forces you to separate "request accepted" from "job completed." The separation is what makes asynchronous processing operationally sound.

Common Pitfalls

  • Starting Celery configuration in Flask but forgetting to run a worker leaves tasks queued forever with no execution.
  • Doing long work directly inside the route blocks the web worker and defeats the point of background processing.
  • Using threads for important production jobs is risky because in-memory work can disappear on process restart.
  • Returning 200 OK with no task tracking makes clients think the work is finished when it has only been scheduled.
  • Ignoring a result backend or status mechanism makes debugging difficult because you cannot inspect task state or failures.

Summary

  • In Flask, production-style asynchronous tasks are usually handled by a worker queue, not by the request thread.
  • Celery plus Redis is a common and practical starting point.
  • Enqueue work with .delay() and return a task id immediately.
  • Run the Flask app and Celery worker as separate processes.
  • Use threads only for small, disposable background jobs where reliability is not critical.

Course illustration
Course illustration

All Rights Reserved.