asyncio
Python concurrency
asynchronous programming
task scheduling
Python event loop

multiple tasks using python asyncio

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

asyncio lets Python run many I/O-bound operations concurrently on a single thread. The key idea is that a coroutine gives control back to the event loop while it is waiting, so other tasks can make progress. When people say "multiple tasks using asyncio", they usually mean creating coroutines, scheduling them as tasks, and coordinating their results safely.

Coroutines, Tasks, and the Event Loop

A coroutine is an async def function. A task is a scheduled wrapper around a coroutine that the event loop can run independently.

You usually work with three pieces:

  • 'async def to define coroutine functions'
  • 'asyncio.create_task() to schedule work'
  • 'await to pause until results are ready'

If a coroutine never reaches an await, it does not cooperate with the event loop and blocks progress just like regular synchronous code.

Running Multiple Tasks Together

The most common pattern is to start several tasks and wait for all of them with asyncio.gather().

python
1import asyncio
2
3async def fetch_value(name: str, delay: float) -> str:
4    await asyncio.sleep(delay)
5    return f"{name} finished"
6
7async def main() -> None:
8    tasks = [
9        asyncio.create_task(fetch_value("task-1", 1.0)),
10        asyncio.create_task(fetch_value("task-2", 0.3)),
11        asyncio.create_task(fetch_value("task-3", 0.6)),
12    ]
13
14    results = await asyncio.gather(*tasks)
15    print(results)
16
17asyncio.run(main())

This starts all three operations immediately and waits until every one completes. For independent I/O work, this is the standard baseline.

Processing Results As Soon As They Finish

If you want early results instead of waiting for the slowest task, use asyncio.as_completed().

python
1import asyncio
2
3async def worker(name: str, delay: float) -> str:
4    await asyncio.sleep(delay)
5    return f"{name} done"
6
7async def main() -> None:
8    coroutines = [
9        worker("A", 0.8),
10        worker("B", 0.2),
11        worker("C", 0.5),
12    ]
13
14    for future in asyncio.as_completed(coroutines):
15        result = await future
16        print(result)
17
18asyncio.run(main())

This is useful for streaming partial progress, updating dashboards, or returning the first acceptable result.

Limiting Concurrency

Creating many tasks is easy, but launching too many network or file operations at once can overwhelm services. A semaphore gives you a simple concurrency cap.

python
1import asyncio
2
3semaphore = asyncio.Semaphore(2)
4
5async def bounded_job(i: int) -> str:
6    async with semaphore:
7        await asyncio.sleep(0.4)
8        return f"job-{i}"
9
10async def main() -> None:
11    tasks = [asyncio.create_task(bounded_job(i)) for i in range(5)]
12    results = await asyncio.gather(*tasks)
13    print(results)
14
15asyncio.run(main())

Only two jobs run inside the semaphore at a time. This pattern matters in real services far more than toy examples suggest.

Error Handling and Cancellation

If one task fails, gather() will raise unless you ask for returned exceptions. That behavior is often correct, but you should choose it deliberately.

python
results = await asyncio.gather(*tasks, return_exceptions=True)

You should also think about cancellation. If a request times out or a program is shutting down, unbounded background tasks can leak work or keep resources open longer than expected.

For new Python versions, TaskGroup is often cleaner for structured concurrency, but create_task() plus gather() remains the most widely understood pattern.

When asyncio Helps and When It Does Not

asyncio is best for I/O-bound workloads:

  • HTTP requests
  • sockets
  • async database drivers
  • message consumers

It does not speed up CPU-heavy loops by itself. For CPU-bound work, use processes or move the heavy function off the event loop.

Common Pitfalls

  • Calling async functions without awaiting or scheduling them. That creates coroutine objects, not running tasks.
  • Mixing blocking functions such as time.sleep() inside coroutines. Use await asyncio.sleep() instead.
  • Launching unbounded numbers of tasks against external systems. Use semaphores or batching.
  • Ignoring cancellation and exception handling in long-running services.
  • Expecting asyncio to make CPU-heavy code faster. It is mainly for concurrent waiting, not parallel computation.

Summary

  • 'asyncio runs many I/O-bound tasks cooperatively on one event loop.'
  • Use create_task() to schedule concurrent work.
  • Use gather() when you need all results and as_completed() when early results matter.
  • Limit concurrency when talking to real external systems.
  • Keep blocking work off the event loop and handle cancellation intentionally.

Course illustration
Course illustration

All Rights Reserved.