Python Asynchronous I/O Walkthrough
January 2017 (assistant professor)
In this 90-minute video series, I walk through a book chapter about asynchronous I/O in Python called A Web Crawler With asyncio Coroutines, which was co-authored by the creator of Python.
I recently stumbled upon an intriguing chapter from a cool book called 500 Lines or Less, co-authored by A. Jesse Jiryu Davis and Guido van Rossum (the creator of Python). This book chapter is called A Web Crawler With asyncio Coroutines.
I've been programming in Python for over a dozen years now but have never explored anything related to concurrency via asynchronous I/O. So I thought it would be fun to record myself walking through this chapter and trying to explain it out loud in my own words. By doing so, I gained a deeper understanding about this fascinating but somewhat difficult-to-grasp topic.
Hopefully these videos are useful as a companion to reading the original text yourself. I covered all sections except for the end where they used the asyncio library, since I was more interested in explaining how things worked under the hood instead of giving a library usage tutorial (which I'm also not qualified to give, since I've never used asyncio in my own work). Enjoy!
Part 1: Concurrency via blocking I/O and threads
Part 2: Introducing nonblocking (async) I/O
Part 3: But async with callback functions is messy
Part 4: Coroutines can eliminate callback messiness, but first need to learn generators
Step through this generator example using Python Tutor.
Part 5: Using generators to build coroutines
Part 6: Cleaning up async code with coroutines
Part 7: Using 'yield from' to let generators call other generators
Step through this generator example using Python Tutor
Part 8: Using 'yield from' to let coroutines call other coroutines, which makes code even cleaner
Note that in reality, you would probably use the asyncio library to implement this technique for reals, instead of manually writing all of this code yourself!
Also, check out How the heck does async/await work in Python 3.5? for more details.
Last modified: 2017-01-09