python
asyncio
threading
concurrency
asynchronous-programming

How to combine python asyncio with threads?

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

asyncio and threads solve different problems, and sometimes a real Python program needs both. asyncio is great for cooperative asynchronous I/O inside one event loop. Threads are useful when you must integrate blocking code or interact with systems that already run work outside the loop.

The Simplest Direction: Async Code Calling Blocking Code

If you already have an async application and one operation still blocks, the cleanest bridge is usually asyncio.to_thread. It runs a regular synchronous function in a worker thread so the event loop can keep making progress.

python
1import asyncio
2import time
3
4
5def blocking_read():
6    time.sleep(2)
7    return "done"
8
9
10async def main():
11    print("starting")
12    result = await asyncio.to_thread(blocking_read)
13    print(result)
14
15
16asyncio.run(main())

This is a good fit for legacy libraries, blocking file operations, or small CPU-light tasks that should not freeze the loop.

Use an Executor When You Need More Control

asyncio.to_thread is convenient, but run_in_executor is still useful when you want to manage the pool explicitly.

python
1import asyncio
2from concurrent.futures import ThreadPoolExecutor
3import time
4
5
6def blocking_job(name: str) -> str:
7    time.sleep(1)
8    return f"finished {name}"
9
10
11async def main():
12    loop = asyncio.get_running_loop()
13
14    with ThreadPoolExecutor(max_workers=4) as pool:
15        results = await asyncio.gather(
16            loop.run_in_executor(pool, blocking_job, "a"),
17            loop.run_in_executor(pool, blocking_job, "b"),
18        )
19        print(results)
20
21
22asyncio.run(main())

Reach for this pattern when thread count, pool reuse, or shutdown behavior matters.

The Reverse Direction: A Thread Submitting Async Work

Sometimes the direction is reversed. You already have a background thread and that thread needs to ask the event loop to run a coroutine. In that case, use asyncio.run_coroutine_threadsafe.

python
1import asyncio
2import threading
3
4
5async def fetch_value():
6    await asyncio.sleep(1)
7    return 42
8
9
10def worker(loop: asyncio.AbstractEventLoop):
11    future = asyncio.run_coroutine_threadsafe(fetch_value(), loop)
12    print(future.result())
13
14
15async def main():
16    loop = asyncio.get_running_loop()
17    thread = threading.Thread(target=worker, args=(loop,))
18    thread.start()
19
20    await asyncio.sleep(2)
21    thread.join()
22
23
24asyncio.run(main())

This matters because most event-loop operations are not thread-safe. A foreign thread should not poke arbitrary loop internals directly.

Keep Clear Ownership of the Event Loop

A safe mental model is:

  • one event loop lives in one thread
  • coroutines run on that loop's thread
  • blocking work may run in worker threads
  • threads cross into the loop only through thread-safe APIs

You can create an event loop in a dedicated thread, but that should be a deliberate integration choice rather than the default. Most applications are simpler when the main thread owns the loop.

Threads Do Not Make CPU-Bound Python Fast

Threads are useful for responsiveness, but they are not a universal answer for CPU-heavy pure Python work. The global interpreter lock means CPU-bound bytecode does not scale the same way I/O-bound work does.

Threads are still fine when:

  • the task mostly waits on external I/O
  • the blocking library releases the global interpreter lock
  • the goal is to keep the event loop responsive

If the workload is truly CPU-heavy, a process pool may be the better tool.

Avoid Common Integration Mistakes

A lot of problems come from mixing the models carelessly. The classic bug is calling time.sleep inside a coroutine, which blocks the whole event-loop thread. Another is calling asyncio.run from code that is already inside a running loop, creating loop ownership confusion instead of solving the real integration problem.

When you mix threads and async code, shared mutable state also needs normal thread-safety rules. asyncio does not make threaded data races disappear.

Common Pitfalls

  • Calling blocking functions directly inside coroutines instead of using to_thread or an executor.
  • Assuming event-loop methods are thread-safe when they are not.
  • Using asyncio.run repeatedly instead of having one clear top-level loop owner.
  • Expecting threads to solve CPU-bound pure Python performance problems.
  • Sharing mutable state between threads and coroutines without synchronization.

Summary

  • Use asyncio.to_thread to run blocking synchronous functions without freezing the event loop.
  • Use run_in_executor when you need more control over the thread pool.
  • Use run_coroutine_threadsafe when another thread must submit work to the loop.
  • Keep one clear owner for each event loop and cross thread boundaries only through safe APIs.
  • For CPU-heavy pure Python work, consider processes instead of threads.

Course illustration
Course illustration

All Rights Reserved.