How to setup a background process using asyncio as part of a module
Interview Questions practice on Codemia
Over 8,000 real interview questions from top companies, searchable by company and role.
Introduction
An asyncio background task should be started by application code that already owns a running event loop. The mistake people usually make is trying to create the task at module import time, which couples import side effects to loop lifecycle and usually breaks in tests, scripts, or alternate runtimes. A module should expose a clean start and stop API instead.
Do Not Start Tasks During Import
This pattern is fragile:
It fails when there is no running loop, and it makes module import perform hidden background work. That is difficult to test and difficult to shut down cleanly.
A Better Pattern: Wrap The Task In A Class
Expose explicit lifecycle methods so the caller decides when the background task starts and stops.
Now the module is reusable because the application controls when the worker exists.
Use It From An Async Entry Point
The caller should start the task from code already running inside the event loop.
This is much easier to reason about than hidden global state.
Keep The Background Loop Focused
A background task should usually do one recurring job such as:
- polling a remote API,
- refreshing cached data,
- draining a queue,
- sending heartbeats.
If the logic becomes large or highly stateful, split the actual unit of work into its own coroutine and let the loop only handle timing and shutdown behavior.
Cancellation Is Part Of The Design
Background tasks need a shutdown path. In asyncio, cancellation is normal, so code should not treat CancelledError as a surprising failure. It is how the event loop signals that the task should stop.
If the task owns a network connection or file handle, put cleanup in finally blocks so it still runs during cancellation.
Integrate Cleanly With Frameworks
Frameworks such as FastAPI, aiohttp, and custom async services already provide startup and shutdown hooks. Those hooks are the right place to call start() and stop().
The rule is simple:
- the framework owns the loop,
- your module owns the worker logic,
- the application wiring connects the two.
That separation keeps the module portable and testable.
Common Pitfalls
- Creating the background task at import time when no loop is running yet.
- Never storing the task reference, which makes shutdown and error handling harder.
- Forgetting to cancel and await the task during application shutdown.
- Catching
CancelledErrorand suppressing it in ways that hide shutdown behavior. - Putting too much unrelated logic into one forever-running coroutine.
Summary
- Start
asynciobackground tasks from application code, not during module import. - Expose explicit lifecycle methods such as
start()andstop(). - Keep a reference to the task so it can be cancelled and awaited cleanly.
- Treat cancellation as a normal shutdown mechanism.
- Let frameworks own the event loop while your module owns the background job logic.
Related reading
- How to setup heterogeneous replication with tungsten?
- How to Simplify Asynchronous Programming in F
- How to sleep the thread in node.js without affecting other threads?
- How to solve the Double-Checked Locking is Broken Declaration in Java?
- How to share faust table between multiple agents or faust timers?
- How to shift a column in Pandas DataFrame
- How to specify a callback method for IAsyncOperation
- How to specify a part of code to run in a particular thread in a multithreaded environment in python?
System design interview problems on this topic
- Design a Distributed Locking System: solution and walkthrough
- Design a Distributed Counter: solution and walkthrough
- Design Ticketmaster: solution and walkthrough
- Design a Flash Sale System: solution and walkthrough
.png&w=3840&q=75)
Tackling System Design Interview Problems
A short course that equips you with the skills to approach system design interviews methodically.
Start the free courseNo course covers this one yet
Basis Lab writes one for you from a sentence about what you want to be able to do, then teaches it and checks you understood. Your first course is free.
Build my courseInterview Questions practice on Codemia
Over 8,000 real interview questions from top companies, searchable by company and role.