asyncio
background process
Python module
asynchronous programming
Python setup

How to setup a background process using asyncio as part of a module

Interview Questions practice on Codemia

Over 8,000 real interview questions from top companies, searchable by company and role.

Browse interview questions

Introduction

An asyncio background task should be started by application code that already owns a running event loop. The mistake people usually make is trying to create the task at module import time, which couples import side effects to loop lifecycle and usually breaks in tests, scripts, or alternate runtimes. A module should expose a clean start and stop API instead.

Do Not Start Tasks During Import

This pattern is fragile:

python
1# bad_module.py
2import asyncio
3
4async def worker():
5    while True:
6        await asyncio.sleep(5)
7        print("tick")
8
9task = asyncio.create_task(worker())

It fails when there is no running loop, and it makes module import perform hidden background work. That is difficult to test and difficult to shut down cleanly.

A Better Pattern: Wrap The Task In A Class

Expose explicit lifecycle methods so the caller decides when the background task starts and stops.

python
1import asyncio
2
3
4class BackgroundWorker:
5    def __init__(self, interval=5):
6        self.interval = interval
7        self._task = None
8        self._stop_event = asyncio.Event()
9
10    async def _run(self):
11        try:
12            while not self._stop_event.is_set():
13                print("background tick")
14                await asyncio.sleep(self.interval)
15        except asyncio.CancelledError:
16            raise
17
18    def start(self):
19        if self._task is not None:
20            return
21        self._stop_event.clear()
22        self._task = asyncio.create_task(self._run())
23
24    async def stop(self):
25        if self._task is None:
26            return
27        self._stop_event.set()
28        self._task.cancel()
29        try:
30            await self._task
31        except asyncio.CancelledError:
32            pass
33        self._task = None

Now the module is reusable because the application controls when the worker exists.

Use It From An Async Entry Point

The caller should start the task from code already running inside the event loop.

python
1import asyncio
2from background_worker import BackgroundWorker
3
4
5async def main():
6    worker = BackgroundWorker(interval=2)
7    worker.start()
8
9    try:
10        await asyncio.sleep(6)
11    finally:
12        await worker.stop()
13
14
15asyncio.run(main())

This is much easier to reason about than hidden global state.

Keep The Background Loop Focused

A background task should usually do one recurring job such as:

  • polling a remote API,
  • refreshing cached data,
  • draining a queue,
  • sending heartbeats.

If the logic becomes large or highly stateful, split the actual unit of work into its own coroutine and let the loop only handle timing and shutdown behavior.

Cancellation Is Part Of The Design

Background tasks need a shutdown path. In asyncio, cancellation is normal, so code should not treat CancelledError as a surprising failure. It is how the event loop signals that the task should stop.

If the task owns a network connection or file handle, put cleanup in finally blocks so it still runs during cancellation.

Integrate Cleanly With Frameworks

Frameworks such as FastAPI, aiohttp, and custom async services already provide startup and shutdown hooks. Those hooks are the right place to call start() and stop().

The rule is simple:

  • the framework owns the loop,
  • your module owns the worker logic,
  • the application wiring connects the two.

That separation keeps the module portable and testable.

Common Pitfalls

  • Creating the background task at import time when no loop is running yet.
  • Never storing the task reference, which makes shutdown and error handling harder.
  • Forgetting to cancel and await the task during application shutdown.
  • Catching CancelledError and suppressing it in ways that hide shutdown behavior.
  • Putting too much unrelated logic into one forever-running coroutine.

Summary

  • Start asyncio background tasks from application code, not during module import.
  • Expose explicit lifecycle methods such as start() and stop().
  • Keep a reference to the task so it can be cancelled and awaited cleanly.
  • Treat cancellation as a normal shutdown mechanism.
  • Let frameworks own the event loop while your module owns the background job logic.

Free course
Beginner
7 lessons
2 hours
Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course
Basis Lab
No course covers this one yet

Basis Lab writes one for you from a sentence about what you want to be able to do, then teaches it and checks you understood. Your first course is free.

Build my course
Interview Questions practice on Codemia

Over 8,000 real interview questions from top companies, searchable by company and role.

Browse interview questions