Lesson 37. asyncio

Mission link: The mission asks for async code that does not stall and an explanation of what a blocking call inside an event loop does. Both come from one fact: there is a single thread, and it only moves when a coroutine yields at an await.
Primary source: The Python Standard Library, asyncio
Prerequisites: Lesson 9, Lesson 10, Lesson 34

Warm-up

1. ▢ Lesson 9: what does a generator do at a yield?

2. ▢ Lesson 35 got an eight-fold speedup from eight threads on eight waits. What does asyncio add?

Know this

async def fetch(i: int) -> int:
    await asyncio.sleep(0.2)         # stands in for a network call
    return i

Calling fetch(1) runs nothing: it builds a coroutine, exactly as calling a generator function builds a generator. Something has to drive it, and that something is the event loop, started by asyncio.run.

await in a loop is sequential

Measured, eight waits of 0.2 seconds:

await in a loop (8 x 0.2s)     1.62s
asyncio.gather                 0.20s
asyncio.TaskGroup              0.20s

results = [await fetch(i) for i in range(8)]         # 1.62s: one at a time
results = await asyncio.gather(*(fetch(i) for i in range(8)))   # 0.20s

await means "suspend until this finishes". It creates no concurrency by itself. Concurrency comes from having several tasks in flight, and a task is created by gather, TaskGroup, or create_task.

This is the most common async bug, and it is invisible: the code is correct, uses async throughout, and runs eight times slower than intended.

TaskGroup over gather

async with asyncio.TaskGroup() as tg:
    tasks = [tg.create_task(fetch(i)) for i in range(8)]
results = [t.result() for t in tasks]

TaskGroup, from Python 3.11, is the one to reach for. The difference is failure behaviour, and it is the whole argument:

• If one task raises, the group cancels the others and waits for them, then raises an ExceptionGroup.
• Nothing is left running when the block exits, so there are no orphaned tasks.
• With gather, a failure leaves the other tasks running unless you pass return_exceptions=True and inspect the results by hand.

Measured, with one task raising ValueError:

TaskGroup raised ExceptionGroup containing ['ValueError']

Which is why except* from lesson 10 exists: a plain except ValueError does not catch a ValueError inside a group.

try:
    async with asyncio.TaskGroup() as tg:
        ...
except* ValueError as eg:
    ...

Cancellation and timeouts

async with asyncio.timeout(0.1):
    await slow()

slow(): saw CancelledError, cleaning up
timeout raised TimeoutError

Two facts to hold together. Cancellation is delivered as CancelledError raised at the await where the coroutine is suspended, so cleanup in a finally or a with runs normally. And asyncio.timeout converts that into TimeoutError for the caller.

The rules that follow:

• Never swallow CancelledError. Catch it only to clean up, then raise. A coroutine that returns normally after being cancelled breaks the shutdown of everything above it.
• A coroutine with no await cannot be cancelled, because there is no point at which the exception can be delivered.
• Cleanup that must complete during cancellation goes in finally, and if it needs to await, wrap it in asyncio.shield or accept that it may be cancelled too.

The blocking call

blocking sleep inside async    1.65s        (time.sleep in eight tasks)
asyncio.to_thread              0.21s

Eight tasks, each calling time.sleep(0.2) instead of await asyncio.sleep(0.2), took as long as doing them one at a time, because there is one thread and a blocking call does not release it to the loop. Nothing raises, nothing warns, and the concurrency silently disappears.

Both runs are the same eight sleeps of the same length, drawn on the same axis. The staircase is what "one thread" looks like when nothing gives it back.

Everything in this category behaves the same way: requests.get, a synchronous database driver, open(...).read() on a slow disk, subprocess.run, hashlib over a large file, a tight computation loop. In a server, one such call in one handler adds its full duration to the latency of every other request in flight.

Three fixes, in order:

1. Use the async library: httpx or aiohttp instead of requests, an async driver instead of a synchronous one.
2. Offload it: await asyncio.to_thread(blocking_fn, *args), which is a thread pool behind an await. Correct for a blocking call you cannot replace.
3. Offload CPU work to a process: loop.run_in_executor(ProcessPoolExecutor(), fn, ...), since a thread would not help per lesson 34.

To find them, run with debug mode on, which logs callbacks that take too long:

PYTHONASYNCIODEBUG=1 python -m myapp

Executing <Task finished name='Task-2' coro=<bad() done ...>> took 0.210 seconds

The rest of the everyday API

Two traps in that table. A task created with create_task and not stored can be garbage collected mid-flight, so keep the reference or use a TaskGroup. And threading.Lock in a coroutine blocks the whole loop rather than yielding, which turns a lock into a stall.

What asyncio does not fix

It does not make CPU-bound code faster: one thread, one interpreter, per lesson 34. It does not remove races, since a task can be suspended at any await, so check-then-act across an await has exactly the bug from lesson 34, without needing another thread. And it does not simplify anything: it colours every function in the call chain, because only a coroutine can await, so adopting it partway through a codebase means duplicating interfaces or bridging with to_thread and asyncio.run.

Choose it when the concurrency is large and the work is waiting. For eight requests, threads are simpler and just as fast.

Practice

1. ▢ Why does this take 1.6 seconds rather than 0.2?

   async def main():
       results = []
       for url in urls:                 # eight urls
           results.append(await fetch(url))
       return results
   

async def main():
    async with asyncio.TaskGroup() as tg:
        tasks = [tg.create_task(fetch(url)) for url in urls]
    return [t.result() for t in tasks]

2. ▢ Find the stall.

   async def handle(request):
       user = await db.fetch_user(request.user_id)
       avatar = requests.get(user.avatar_url).content     # 300 ms
       return render(user, avatar)
   

3. ▢ TaskGroup or gather?

   • a) Fetch three resources; if any fails, the request fails
   • b) Notify twelve webhooks; failures are logged and ignored
   • c) Two queries whose results are both required
   • d) A background metrics flusher that runs for the process's life

4. ▢ What is wrong with this cleanup?

   async def worker():
       try:
           await do_work()
       except asyncio.CancelledError:
           log.info("cancelled")
           return
   

except asyncio.CancelledError:
    log.info("cancelled")
    raise

5. ▢ Does this race, in a single-threaded event loop?

   async def get_or_create(key):
       if key not in cache:
           cache[key] = await build(key)
       return cache[key]
   

unsafe: builds for one key = 8
with an asyncio.Lock: builds = 1

locks: dict[str, asyncio.Lock] = defaultdict(asyncio.Lock)

async def get_or_create(key):
    if key in cache:
        return cache[key]
    async with locks[key]:
        if key not in cache:                 # check again, holding the lock
            cache[key] = await build(key)
    return cache[key]

6. ▢ A team proposes converting a synchronous service to asyncio for performance. It handles 50 concurrent requests, each spending 95 per cent of its time on database queries. Advise them.

Real-world reps

• [ ] Time an await inside a for loop against the same work in a TaskGroup. The factor is the number of items, and seeing it once fixes the habit.
• [ ] Run an async program with PYTHONASYNCIODEBUG=1 and read what it reports. Every slow callback is a blocking call.
• [ ] Grep an async codebase for requests., time.sleep, open(, and subprocess.run inside async def. Each is a stall.
• [ ] Find every except asyncio.CancelledError and confirm each one re-raises.
• [ ] Find every create_task and confirm the task is referenced somewhere that outlives it.
• [ ] Tomorrow: write the check-then-act across an await from practice 5 and observe the duplicate build, so the single-threaded-is-not-atomic point lands.

Going further

• asyncio: the whole library, with the low-level parts marked as such
• Coroutines and Tasks: TaskGroup, gather, timeout, to_thread, and cancellation
• Developing with asyncio: debug mode, and the documented list of what blocks the loop
• PEP 654, Exception Groups and except*: how a TaskGroup reports several failures
• contextvars: per-task context, since a thread-local is shared by every task
• Resources

Not landing? Reread the primary source at the top, since this lesson compresses it and compression is where understanding leaks. Check the glossary for any term that felt slippery.

If the lesson itself is unclear rather than the material, that is a defect: open an issue.