multiple tasks using python asyncio
Interview Questions practice on Codemia
Over 8,000 real interview questions from top companies, searchable by company and role.
Introduction
asyncio lets Python run many I/O-bound operations concurrently on a single thread. The key idea is that a coroutine gives control back to the event loop while it is waiting, so other tasks can make progress. When people say "multiple tasks using asyncio", they usually mean creating coroutines, scheduling them as tasks, and coordinating their results safely.
Coroutines, Tasks, and the Event Loop
A coroutine is an async def function. A task is a scheduled wrapper around a coroutine that the event loop can run independently.
You usually work with three pieces:
- '
async defto define coroutine functions' - '
asyncio.create_task()to schedule work' - '
awaitto pause until results are ready'
If a coroutine never reaches an await, it does not cooperate with the event loop and blocks progress just like regular synchronous code.
Running Multiple Tasks Together
The most common pattern is to start several tasks and wait for all of them with asyncio.gather().
This starts all three operations immediately and waits until every one completes. For independent I/O work, this is the standard baseline.
Processing Results As Soon As They Finish
If you want early results instead of waiting for the slowest task, use asyncio.as_completed().
This is useful for streaming partial progress, updating dashboards, or returning the first acceptable result.
Limiting Concurrency
Creating many tasks is easy, but launching too many network or file operations at once can overwhelm services. A semaphore gives you a simple concurrency cap.
Only two jobs run inside the semaphore at a time. This pattern matters in real services far more than toy examples suggest.
Error Handling and Cancellation
If one task fails, gather() will raise unless you ask for returned exceptions. That behavior is often correct, but you should choose it deliberately.
You should also think about cancellation. If a request times out or a program is shutting down, unbounded background tasks can leak work or keep resources open longer than expected.
For new Python versions, TaskGroup is often cleaner for structured concurrency, but create_task() plus gather() remains the most widely understood pattern.
When asyncio Helps and When It Does Not
asyncio is best for I/O-bound workloads:
- HTTP requests
- sockets
- async database drivers
- message consumers
It does not speed up CPU-heavy loops by itself. For CPU-bound work, use processes or move the heavy function off the event loop.
Common Pitfalls
- Calling
asyncfunctions without awaiting or scheduling them. That creates coroutine objects, not running tasks. - Mixing blocking functions such as
time.sleep()inside coroutines. Useawait asyncio.sleep()instead. - Launching unbounded numbers of tasks against external systems. Use semaphores or batching.
- Ignoring cancellation and exception handling in long-running services.
- Expecting
asyncioto make CPU-heavy code faster. It is mainly for concurrent waiting, not parallel computation.
Summary
- '
asyncioruns many I/O-bound tasks cooperatively on one event loop.' - Use
create_task()to schedule concurrent work. - Use
gather()when you need all results andas_completed()when early results matter. - Limit concurrency when talking to real external systems.
- Keep blocking work off the event loop and handle cancellation intentionally.
Related reading
- Multiplexing callbacks
- multiprocess or threading in python?
- Multiprocessing causes Python to crash and gives an error may have been in progress in another thread when fork was called
- Multiprocessing How to use Pool.map on a function defined in a class?
- Multiple variables in a 'with' statement?
- Multiprocessing example giving AttributeError in Jupyter Notebook
- Multiprocessing on a model with data frame as input
- Multiprocessing or Multithreading?
.png&w=3840&q=75)
Tackling System Design Interview Problems
A short course that equips you with the skills to approach system design interviews methodically.
Start the free courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
Interview Questions practice on Codemia
Over 8,000 real interview questions from top companies, searchable by company and role.