Nelson's log

A simpler explanation of asyncio

Reflecting on my previous effort to understand asyncio, I think I have a simple summary:

asyncio is a cooperative multitasking system for Python based on generators. You implement your program as generator functions that yield control periodically. The asyncio event loop manages running all of your generators.

That’s it, it’s really simple. This understanding lets me build on my existing knowledge about generators / coroutines and understand what asyncio is doing.

What’s confusing is all the syntactic sugar and extra library stuff in asyncio gets in the way of understanding the simple thing. Also the Python docs are not written from the point of view of “explain the basic idea”. And the API is still in transition; in Python 3.5 they are no longer generators, they are coroutines, but the principals are the same.

Here’s a really simple program:

import asyncio

def sleeper(loop):
    for i in range(10):
        print(i, loop.time())
        yield from asyncio.sleep(0.1)

loop = asyncio.get_event_loop()
task = sleeper(loop)

That’s a toy example. In real work, you have multiple tasks running simultaneously. And instead of yielding on a sleep you yield on something more useful, network IO or something.

The confusing part is the line 10, task = sleeper(loop). Naively you’d expect that to invoke the sleeper function and do the work right then. But because it’s been decorated as a coroutine, really what calling that function does is return a generator object that encapsulates calling the function. We then pass that generator to the event loop to ask that asyncio call it for us. (In Python 3.5 using async def and await, that object is no longer a generator, it’s a member of the new type, coroutine.)

There’s a lot more complexity of course. I’m still confused about the melange of coroutine, Task, and Future, and when to use which. Combining multiple coroutines in various ways can get complex, although once you commit to the model it is easier to reason about than thread locking. And then there’s a lot of auxiliary libraries that make asyncio useful, things that help you yield from network IO or external processes or whatever. Not to mention libraries on top of asyncio like aiohttp.

But the core concept there is really simple. Generator functions that cooperatively multitask. That’s the magic, there’s not too much of it really.