Catch exception in asyncio gather

asyncio

Python

exception handling

gather

asynchronous programming

Catch exception in asyncio gather

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Start Practicing Learn More

Introduction

asyncio.gather() runs multiple coroutines concurrently and collects their results. By default, if any task raises an exception, gather cancels the remaining tasks and re-raises the first exception. The return_exceptions=True parameter changes this behavior to return exceptions as values in the result list instead of raising them.

Default Behavior (Raises Exceptions)

python

1import asyncio
2
3async def success():
4    await asyncio.sleep(1)
5    return "ok"
6
7async def failure():
8    await asyncio.sleep(0.5)
9    raise ValueError("something went wrong")
10
11async def main():
12    try:
13        results = await asyncio.gather(success(), failure())
14    except ValueError as e:
15        print(f"Caught: {e}")  # Caught: something went wrong
16
17asyncio.run(main())

The first exception cancels other tasks and propagates up.

Using return_exceptions=True

With return_exceptions=True, exceptions are returned as values alongside successful results:

python

1async def main():
2    results = await asyncio.gather(
3        success(),
4        failure(),
5        success(),
6        return_exceptions=True
7    )
8
9    for i, result in enumerate(results):
10        if isinstance(result, Exception):
11            print(f"Task {i} failed: {result}")
12        else:
13            print(f"Task {i} succeeded: {result}")
14
15# Task 0 succeeded: ok
16# Task 1 failed: something went wrong
17# Task 2 succeeded: ok

All tasks run to completion regardless of individual failures.

Handling Mixed Results

python

1async def fetch_data(url):
2    await asyncio.sleep(0.5)
3    if "bad" in url:
4        raise ConnectionError(f"Cannot connect to {url}")
5    return f"Data from {url}"
6
7async def main():
8    urls = ["https://api.good.com", "https://api.bad.com", "https://api.ok.com"]
9
10    results = await asyncio.gather(
11        *[fetch_data(url) for url in urls],
12        return_exceptions=True
13    )
14
15    successes = [(url, r) for url, r in zip(urls, results) if not isinstance(r, Exception)]
16    failures = [(url, r) for url, r in zip(urls, results) if isinstance(r, Exception)]
17
18    print(f"Succeeded: {len(successes)}")
19    for url, data in successes:
20        print(f"  {url}: {data}")
21
22    print(f"Failed: {len(failures)}")
23    for url, err in failures:
24        print(f"  {url}: {err}")
25
26asyncio.run(main())

Per-Task Try/Except

Wrap individual coroutines for fine-grained error handling:

python

1async def safe_fetch(url):
2    try:
3        return await fetch_data(url)
4    except Exception as e:
5        return {"error": str(e), "url": url}
6
7async def main():
8    urls = ["https://api.good.com", "https://api.bad.com"]
9    results = await asyncio.gather(*[safe_fetch(url) for url in urls])
10    # All results are either data or error dicts — no exceptions raised

asyncio.gather vs TaskGroup (Python 3.11+)

Python 3.11 introduced TaskGroup as a structured alternative:

python

1async def main():
2    try:
3        async with asyncio.TaskGroup() as tg:
4            task1 = tg.create_task(success())
5            task2 = tg.create_task(failure())
6            task3 = tg.create_task(success())
7    except* ValueError as eg:
8        for exc in eg.exceptions:
9            print(f"Caught: {exc}")

Feature	`asyncio.gather`	`asyncio.TaskGroup`
Python version	3.4+	3.11+
Exception handling	First exception or `return_exceptions`	`ExceptionGroup` with `except*`
Cancellation	Cancels remaining on first error	Cancels remaining on first error
Structured concurrency	No	Yes

Timeout with gather

python

1async def main():
2    try:
3        results = await asyncio.wait_for(
4            asyncio.gather(
5                fetch_data("https://api.slow.com"),
6                fetch_data("https://api.fast.com"),
7                return_exceptions=True
8            ),
9            timeout=5.0
10        )
11    except asyncio.TimeoutError:
12        print("Overall timeout exceeded")

Combining gather with Semaphore

Limit concurrency when gathering many tasks:

python

1sem = asyncio.Semaphore(10)  # max 10 concurrent
2
3async def limited_fetch(url):
4    async with sem:
5        return await fetch_data(url)
6
7async def main():
8    urls = [f"https://api.example.com/{i}" for i in range(100)]
9    results = await asyncio.gather(
10        *[limited_fetch(url) for url in urls],
11        return_exceptions=True
12    )

Common Pitfalls

Task Continuation: With return_exceptions=True, even if one task fails, others proceed and complete, potentially returning either results or exceptions. This is the whole point — but check each result individually.
Careful Inspection: When using return_exceptions=True, carefully inspect the result list to handle exceptions appropriately, as they are part of the output. Use isinstance(result, Exception) to detect failures.
Performance Impact: Proper error handling encourages better performance and reliability, especially in web servers or applications making diverse network calls.
Cancellation propagation: Without return_exceptions=True, the first exception causes gather to cancel remaining tasks. Cancelled tasks raise CancelledError, which may mask the original exception.
Fire and forget: Creating tasks with asyncio.gather and not awaiting the result means exceptions are silently lost. Always await gather.
Exception types: return_exceptions=True returns all exception types, including CancelledError. Filter appropriately if you need to distinguish cancellation from real errors.

Summary

By default, asyncio.gather re-raises the first exception and cancels remaining tasks
Use return_exceptions=True to collect all results (successes and exceptions) without cancellation
Check results with isinstance(result, Exception) to separate successes from failures
Use TaskGroup (Python 3.11+) for structured concurrency with except* handling
Wrap individual coroutines in try/except for per-task error handling without return_exceptions