Python Asynchronous I/O Walkthrough

I recently stumbled upon an intriguing chapter from a cool book called 500 Lines or Less, co-authored by A. Jesse Jiryu Davis and Guido van Rossum (the creator of Python). This book chapter is called A Web Crawler With asyncio Coroutines.

I've been programming in Python for over a dozen years now but have never explored anything related to concurrency via asynchronous I/O. So I thought it would be fun to record myself walking through this chapter and trying to explain it out loud in my own words. By doing so, I gained a deeper understanding about this fascinating but somewhat difficult-to-grasp topic.

Hopefully these videos are useful as a companion to reading the original text yourself. I covered all sections except for the end where they used the asyncio library, since I was more interested in explaining how things worked under the hood instead of giving a library usage tutorial (which I'm also not qualified to give, since I've never used asyncio in my own work). Enjoy!

(full YouTube playlist)

Part 1: Concurrency via blocking I/O and threads

Part 2: Introducing nonblocking (async) I/O

Part 3: But async with callback functions is messy

Part 4: Coroutines can eliminate callback messiness, but first need to learn generators

Step through this generator example using Python Tutor.

Part 5: Using generators to build coroutines

Part 6: Cleaning up async code with coroutines

Part 7: Using 'yield from' to let generators call other generators

Step through this generator example using Python Tutor

Part 8: Using 'yield from' to let coroutines call other coroutines, which makes code even cleaner

Note that in reality, you would probably use the asyncio library to implement this technique for reals, instead of manually writing all of this code yourself!

Also, check out How the heck does async/await work in Python 3.5? for more details.

The end!