multiple tasks using python asyncio
Master System Design with Codemia
Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.
Introduction
asyncio lets Python run many I/O-bound operations concurrently on a single thread. The key idea is that a coroutine gives control back to the event loop while it is waiting, so other tasks can make progress. When people say "multiple tasks using asyncio", they usually mean creating coroutines, scheduling them as tasks, and coordinating their results safely.
Coroutines, Tasks, and the Event Loop
A coroutine is an async def function. A task is a scheduled wrapper around a coroutine that the event loop can run independently.
You usually work with three pieces:
- '
async defto define coroutine functions' - '
asyncio.create_task()to schedule work' - '
awaitto pause until results are ready'
If a coroutine never reaches an await, it does not cooperate with the event loop and blocks progress just like regular synchronous code.
Running Multiple Tasks Together
The most common pattern is to start several tasks and wait for all of them with asyncio.gather().
This starts all three operations immediately and waits until every one completes. For independent I/O work, this is the standard baseline.
Processing Results As Soon As They Finish
If you want early results instead of waiting for the slowest task, use asyncio.as_completed().
This is useful for streaming partial progress, updating dashboards, or returning the first acceptable result.
Limiting Concurrency
Creating many tasks is easy, but launching too many network or file operations at once can overwhelm services. A semaphore gives you a simple concurrency cap.
Only two jobs run inside the semaphore at a time. This pattern matters in real services far more than toy examples suggest.
Error Handling and Cancellation
If one task fails, gather() will raise unless you ask for returned exceptions. That behavior is often correct, but you should choose it deliberately.
You should also think about cancellation. If a request times out or a program is shutting down, unbounded background tasks can leak work or keep resources open longer than expected.
For new Python versions, TaskGroup is often cleaner for structured concurrency, but create_task() plus gather() remains the most widely understood pattern.
When asyncio Helps and When It Does Not
asyncio is best for I/O-bound workloads:
- HTTP requests
- sockets
- async database drivers
- message consumers
It does not speed up CPU-heavy loops by itself. For CPU-bound work, use processes or move the heavy function off the event loop.
Common Pitfalls
- Calling
asyncfunctions without awaiting or scheduling them. That creates coroutine objects, not running tasks. - Mixing blocking functions such as
time.sleep()inside coroutines. Useawait asyncio.sleep()instead. - Launching unbounded numbers of tasks against external systems. Use semaphores or batching.
- Ignoring cancellation and exception handling in long-running services.
- Expecting
asyncioto make CPU-heavy code faster. It is mainly for concurrent waiting, not parallel computation.
Summary
- '
asyncioruns many I/O-bound tasks cooperatively on one event loop.' - Use
create_task()to schedule concurrent work. - Use
gather()when you need all results andas_completed()when early results matter. - Limit concurrency when talking to real external systems.
- Keep blocking work off the event loop and handle cancellation intentionally.

