Python
asyncio
Jupyter Notebook
event loop
async programming

asyncio.run cannot be called from a running event loop when using Jupyter Notebook

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

When you call asyncio.run() inside a Jupyter notebook, Python raises RuntimeError: This event loop is already running. The reason is straightforward: Jupyter's kernel already manages its own asyncio event loop for interactive execution, and asyncio.run() tries to create a second top-level loop, which asyncio does not allow.

The fix in most cases is to stop using asyncio.run() in notebooks and instead await your coroutines directly at the cell level. Jupyter supports top-level await natively, making it the cleanest solution.

Why the Error Happens

asyncio.run() is designed as the single entry point for a standalone async program. Under the hood, it does three things:

  1. Creates a new event loop
  2. Runs the given coroutine to completion on that loop
  3. Closes the loop when the coroutine finishes
python
1# This works in a normal Python script
2import asyncio
3
4async def main():
5    await asyncio.sleep(1)
6    return "done"
7
8if __name__ == "__main__":
9    result = asyncio.run(main())
10    print(result)

In a Jupyter notebook, the IPython kernel starts an event loop during initialization and keeps it running for the entire session. When your cell calls asyncio.run(), it attempts step 1 (create a new loop) while the kernel's loop is already active. Python's asyncio explicitly forbids nested loops, so it raises the error.

The same issue can appear in other environments that run their own event loop, including certain GUI frameworks (Tkinter, Qt), web servers (FastAPI/Uvicorn during testing), and some IDE debuggers.

Modern Jupyter notebooks (IPython 7.0+ with Python 3.8+) support top-level await directly in cells:

python
1import asyncio
2
3async def fetch_data():
4    await asyncio.sleep(0.5)
5    return {"status": "ok", "records": 42}
6
7# Just await the coroutine directly - no asyncio.run() needed
8result = await fetch_data()
9print(result)

This works because Jupyter hooks into the already-running event loop and schedules the coroutine on it. There is no conflict because no second loop is created.

Multiple Coroutines in One Cell

You can gather multiple coroutines in a single cell:

python
1import asyncio
2
3async def fetch_users():
4    await asyncio.sleep(0.3)
5    return ["alice", "bob"]
6
7async def fetch_orders():
8    await asyncio.sleep(0.2)
9    return [{"id": 1}, {"id": 2}]
10
11users, orders = await asyncio.gather(fetch_users(), fetch_orders())
12print(f"Users: {users}")
13print(f"Orders: {orders}")

Both coroutines run concurrently on Jupyter's existing loop.

Solution 2: asyncio.create_task() for Background Work

When you want work to run concurrently without blocking the cell, create a task:

python
1import asyncio
2
3async def background_job(name, duration):
4    print(f"[{name}] started")
5    await asyncio.sleep(duration)
6    print(f"[{name}] finished")
7    return f"{name} result"
8
9task = asyncio.create_task(background_job("data-sync", 2.0))
10
11# Do other work in the cell while the task runs
12print("Task is running in the background")
13
14# Later, await the result when you need it
15result = await task
16print(result)

This is useful for long-running operations where you want to check progress or run other cells while waiting.

Solution 3: nest_asyncio (Compatibility Fallback)

When you cannot modify the code that calls asyncio.run() (for example, a third-party library), nest_asyncio patches the event loop to allow re-entrant execution:

python
1import nest_asyncio
2nest_asyncio.apply()
3
4# Now asyncio.run() works even inside Jupyter
5import asyncio
6
7async def main():
8    await asyncio.sleep(0.1)
9    return "patched"
10
11result = asyncio.run(main())
12print(result)

Install it with:

bash
pip install nest_asyncio

When nest_asyncio Is the Right Choice

ScenarioUse nest_asyncio?
You control the notebook codeNo, use top-level await
A library internally calls asyncio.run()Yes
You are porting a script to a notebook temporarilyMaybe, but refactoring to await is better
You are running tests that use asyncio.run() in notebooksYes
Production codeNo, fix the architecture instead

nest_asyncio is a compatibility shim, not an architectural solution. Prefer restructuring your code when possible.

Writing Code That Works in Both Scripts and Notebooks

The cleanest approach is to keep async logic in coroutines and choose the runner based on the environment:

python
1import asyncio
2
3async def process_data():
4    """Pure async logic - no environment-specific code here."""
5    await asyncio.sleep(0.5)
6    data = [1, 2, 3, 4, 5]
7    return [x * 2 for x in data]

In a script:

python
1# run_script.py
2import asyncio
3from mymodule import process_data
4
5if __name__ == "__main__":
6    result = asyncio.run(process_data())
7    print(result)

In a notebook cell:

python
1from mymodule import process_data
2
3result = await process_data()
4print(result)

This pattern keeps the business logic reusable and moves the environment-specific entry point out of the coroutine.

Detecting the Environment Programmatically

If you need a single entry point that works in both contexts:

python
1import asyncio
2
3async def main():
4    await asyncio.sleep(0.1)
5    return "result"
6
7def run():
8    try:
9        loop = asyncio.get_running_loop()
10    except RuntimeError:
11        # No loop running - safe to use asyncio.run()
12        return asyncio.run(main())
13    else:
14        # Loop already running (Jupyter, etc.) - create a task
15        import nest_asyncio
16        nest_asyncio.apply()
17        return asyncio.run(main())

This works but adds complexity. For library authors, it is better to expose the coroutine directly and let callers decide how to run it.

Debugging Library-Caused Loop Conflicts

Some notebook errors appear only after calling a third-party library that internally uses asyncio. Common culprits include HTTP client libraries, database drivers, and message queue clients.

The debugging process:

  1. Check if the library exposes an async/awaitable API alongside the sync one
  2. If it does, use the async API with await instead of wrapping the sync API with asyncio.run()
  3. If it only has a sync API that internally uses asyncio.run(), apply nest_asyncio before importing the library
python
1# Instead of this (breaks in notebooks):
2import httpx
3response = asyncio.run(httpx.AsyncClient().get("https://example.com"))
4
5# Do this:
6import httpx
7
8async with httpx.AsyncClient() as client:
9    response = await client.get("https://example.com")
10    print(response.status_code)

Comparison of Solutions

ApproachComplexityWhen to Use
Top-level awaitLowDefault choice for all notebook async code
asyncio.create_task()LowConcurrent background work in notebooks
nest_asyncioMediumThird-party code that calls asyncio.run() internally
Environment detectionHighLibraries that must work everywhere without modification
Refactoring to coroutinesMediumLong-term solution for portable async code

Common Pitfalls

Calling asyncio.run() inside a notebook cell because a tutorial or Stack Overflow answer used it in a script context is the most common trigger. Tutorials written for scripts are not directly portable to notebooks.

Using nest_asyncio as the default solution when top-level await would be simpler and cleaner adds an unnecessary dependency and hides the real issue.

Mixing notebook-style await and manual loop control (loop.run_until_complete()) in the same session can cause subtle ordering bugs and unexpected behavior.

Assuming the error means the coroutine itself is broken leads to wasted debugging time. The coroutine is usually fine. The problem is how it is being invoked.

Forgetting that asyncio.create_task() requires an already-running loop to work means it functions perfectly in notebooks but fails in plain scripts without asyncio.run() wrapping it.

Summary

  • Jupyter already runs an event loop, so asyncio.run() conflicts with it by trying to create a second one.
  • In notebooks, use top-level await to run coroutines. This is the simplest and most Pythonic solution.
  • Use asyncio.create_task() when you need concurrent work within a notebook session.
  • Reserve nest_asyncio for compatibility with third-party libraries that internally call asyncio.run().
  • Keep async business logic in coroutines and vary only the entry point between scripts and notebooks.
  • When a library causes the error, check whether it offers an async API you can await directly.

Course illustration
Course illustration

All Rights Reserved.