Python Notebooks Async

Overview

Notebook kernels own the event loop; async code must cooperate with that ownership rather than fight it. This skill covers orchestration patterns, top-level await, and compatibility constraints for .ipynb and #%% workflows.

Treat these recommendations as preferred defaults. When project constraints require deviation, call out tradeoffs and compensating controls.

When to Use

• asyncio.run() raises RuntimeError inside a notebook cell.
• Event-loop conflicts when mixing async libraries in Jupyter.
• Porting async scripts into notebook workflows.
• Orchestrating concurrent tasks (gather, TaskGroup) in IPython kernels.
• Deciding where to place reusable async logic across notebook/module boundaries.

When NOT to Use

• Pure script or service code with no notebook involvement — see python-concurrency-performance.
• Synchronous notebook workflows with no async needs.
• General asyncio API design outside notebook contexts — see python-runtime-operations.

Quick Reference

• Treat notebook kernels as loop-owned environments; never create a competing loop.
• Use top-level await instead of asyncio.run() in notebook cells.
• Orchestrate concurrent work with asyncio.gather() or asyncio.TaskGroup.
• Keep reusable async logic in regular .py modules, imported into notebooks.
• Use nest_asyncio only as a constrained compatibility fallback, not a default.
• Avoid fire-and-forget tasks — always await or collect results explicitly.

Common Mistakes

• Calling `asyncio.run()` in a notebook cell.

The kernel already runs a loop; asyncio.run() tries to start a second one and raises RuntimeError. Use await directly instead.

• Applying `nest_asyncio` globally by default.

It patches the loop to allow reentrant calls but masks design problems and can hide subtle concurrency bugs. Reserve it for legacy compatibility.

• Defining async helpers inline in cells instead of modules.

Inline definitions are lost on kernel restart and cannot be tested outside the notebook. Extract to .py files.

• Ignoring returned tasks or coroutines.

Calling an async function without await silently produces a never-executed coroutine object, with no error until results are missing downstream.

• Mixing blocking I/O with async in the same cell.

Synchronous calls like requests.get() block the event loop, starving concurrent tasks. Use aiohttp, httpx, or asyncio.to_thread().

Scope Note

• Treat these recommendations as preferred defaults for common cases, not universal rules.
• If a default conflicts with project constraints or worsens the outcome, suggest a better-fit alternative and explain why it is better for this case.
• When deviating, call out tradeoffs and compensating controls (tests, observability, migration, rollback).

Invocation Notice

• Inform the user when this skill is being invoked by name: python-design-modularity.

References

• references/notebooks-async.md