Philip Guo (Phil Guo, Philip J. Guo, Philip Jia Guo, pgbovine)

Interactive Systems for Learning Programming at Scale

(One-hour seminar talk)

Here is a one-hour invited talk that I gave at the Stanford Human-Computer Interaction Seminar about my main line of research from 2013–2016:

And here is a similar talk at the UC Berkeley School of Information. This one was for a faculty job interview, so it has more extensive discussions of my vision for future work:

This talk covers these four papers:

Online Python Tutor: Embeddable Web-Based Program Visualization for CS Education. Philip J. Guo. ACM Technical Symposium on Computer Science Education (SIGCSE), 2013.
Python Tutor lets anyone write code in their web browser and visualize step-by-step run-time state.
This paper presents Online Python Tutor, a web-based program visualization tool for Python, which is becoming a popular language for teaching introductory CS courses. Using this tool, teachers and students can write Python programs directly in the web browser (without installing any plugins), step forwards and backwards through execution to view the run-time state of data structures, and share their program visualizations on the web.

In the past three years, over 200,000 people have used Online Python Tutor to visualize their programs. In addition, instructors in a dozen universities such as UC Berkeley, MIT, the University of Washington, and the University of Waterloo have used it in their CS1 courses. Finally, Online Python Tutor visualizations have been embedded within three web-based digital Python textbook projects, which collectively attract around 16,000 viewers per month and are being used in at least 25 universities. Online Python Tutor is free and open source software, available at
 author = {Guo, Philip J.},
 title = {{Online Python Tutor}: Embeddable Web-based Program Visualization for {CS} Education},
 booktitle = {Proceedings of the 44th ACM Technical Symposium on Computer Science Education},
 series = {SIGCSE '13},
 year = {2013},
 isbn = {978-1-4503-1868-6},
 location = {Denver, Colorado, USA},
 pages = {579--584},
 numpages = {6},
 url = {},
 doi = {10.1145/2445196.2445368},
 acmid = {2445368},
 publisher = {ACM},
 address = {New York, NY, USA},
 keywords = {CS1, program visualization, python},
Codechella: Multi-User Program Visualizations for Real-Time Tutoring and Collaborative Learning. Philip J. Guo, Jeffery White, Renan Zanelatto. IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), 2015.
Codechella lets multiple people write code, visualize run-time state, debug, and chat in real time.
An effective way to learn computer programming is to sit side-by-side in front of the same computer with a tutor or peer, write code together, and then discuss what happens as the code executes. To bring this kind of in-person interaction to an online setting, we have developed Codechella, a multi-user Web-based program visualization system that enables multiple people to collaboratively write code together, explore an automatically-generated visualization of its execution state using multiple mouse cursors, and chat via an embedded text box. In the past nine months of live deployment on an educational website, people from 296 cities across 40 countries have started 299 Codechella sessions for both tutoring and collaborative learning. 57% of sessions connected participants from different cities. 69% of actions were visualization interactions, which indicates high engagement with program visualizations. Finally, participants showed signs of learning at the lower three levels of Bloom's taxonomy: remembering, understanding, and applying knowledge.
  author={Guo, Philip J. and White, Jeffery and Zanelatto, Renan},
  title={Codechella: Multi-user program visualizations for real-time tutoring and collaborative learning},
  booktitle = {Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)},
  series = {VL/HCC '15},
Codeopticon: Real-Time, One-To-Many Human Tutoring for Computer Programming. Philip J. Guo. ACM Symposium on User Interface Software and Technology (UIST), 2015.
Codeopticon lets instructors monitor and chat with dozens of students at once while they are coding.
One-on-one tutoring from a human expert is an effective way for novices to overcome learning barriers in complex domains such as computer programming. But there are usually far fewer experts than learners. To enable a single expert to help more learners at once, we built Codeopticon, an interface that enables a programming tutor to monitor and chat with dozens of learners in real time. Each learner codes in a workspace that consists of an editor, compiler, and visual debugger. The tutor sees a real-time view of each learner's actions on a dashboard, with each learner's workspace summarized in a tile. At a glance, the tutor can see how learners are editing and debugging their code, and what errors they are encountering. The dashboard automatically reshuffles tiles so that the most active learners are always in the tutor's main field of view. When the tutor sees that a particular learner needs help, they can open an embedded chat window to start a one-on-one conversation. A user study showed that 8 first-time Codeopticon users successfully tutored anonymous learners from 54 countries in a naturalistic online setting. On average, in a 30-minute session, each tutor monitored 226 learners, started 12 conversations, exchanged 47 chats, and helped 2.4 learners.
 author = {Guo, Philip J.},
 title = {Codeopticon: Real-Time, One-To-Many Human Tutoring for Computer Programming},
 booktitle = {Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology},
 series = {UIST '15},
 year = {2015},
 isbn = {978-1-4503-3779-3},
 pages = {599--608},
 numpages = {10},
 url = {},
 doi = {10.1145/2807442.2807469},
 acmid = {2807469},
 publisher = {ACM},
 address = {New York, NY, USA},
 keywords = {computer programming, learning at scale, remote tutoring}
Codepourri: Creating Visual Coding Tutorials Using A Volunteer Crowd Of Learners. Mitchell Gordon and Philip J. Guo. IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), 2015.
Codepourri lets an anonymous online crowd create step-by-step coding tutorials together.
A popular way to learn is by studying written tutorials. However, tutorials for computer programming can be tedious to create, since a static text-based format cannot visualize what happens as code executes. We created a system called Codepourri that enables people to easily create visual coding tutorials by annotating steps in an automatically-generated program visualization. Using Codepourri, we developed a crowdsourcing workflow where learners who are visiting an educational website collectively create a tutorial by annotating individual steps and then voting on the best annotations. Since there are far more learners than experts, using learners as a crowd is a potentially more scalable way of creating tutorials. Our experiments with 4 expert judges and 101 learners adding 145 raw annotations to Python code show the learner crowd's annotations to be accurate, informative, and containing some insights that even experts missed.
  author={Gordon, Mitchell and Guo, Philip J.},
  title={Codepourri: Creating visual coding tutorials using a volunteer crowd of learners},
  booktitle = {Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)},
  series = {VL/HCC '15},
Created: 2016-02-19
Last modified: 2016-02-19
Related pages tagged as research: