Philip Guo (Phil Guo, Philip J. Guo, Philip Jia Guo, pgbovine)

Omnicode: A Novice-Oriented Live Programming Environment with Always-On Run-Time Value Visualizations

research paper summary
Omnicode: A Novice-Oriented Live Programming Environment with Always-On Run-Time Value Visualizations. Hyeonsu Kang and Philip J. Guo. ACM Symposium on User Interface Software and Technology (UIST), 2017.
Visualizations of run-time program state help novices form proper mental models and debug their code. We push this technique to the extreme by posing the following question: What if a live programming environment for an imperative language always displays the entire history of all run-time values for all program variables all the time? To explore this question, we built a prototype live IDE called Omnicode ("Omniscient Code") that continually runs the user's Python code and uses a scatterplot matrix to visualize the entire history of all of its numerical values, along with meaningful numbers derived from other data types. To filter the visualizations and hone in on specific points of interest, the user can brush and link over the scatterplots or select portions of code. They can also zoom in to view detailed stack and heap visualizations at each execution step. An exploratory study on 10 novice programmers discovered that they found Omnicode to be useful for debugging, forming mental models, explaining their code to others, and discovering moments of serendipity that would not have been likely within an ordinary IDE.
@inproceedings{KangUIST2017,
 author = {Kang, Hyeonsu and Guo, Philip J.},
 title = {Omnicode: A Novice-Oriented Live Programming Environment with Always-On Run-Time Value Visualizations},
 booktitle = {Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology},
 series = {UIST '17},
 year = {2017},
 publisher = {ACM},
 address = {New York, NY, USA},
}

One of the most fundamental skills that programming students need to master is being able to form mental models of how static pieces of source code correspond to dynamic run-time actions inside of a computer. Without this basic core foundation, it's impossible to become a proficient programmer. For instance, consider this tiny Python example: x=[1,2,3]; y=x; x[0]=100. What's the value of y[0] after this code runs? To be able to answer even this simple question, a student must develop a viable mental model of how the Python = operator affects lists and integers.

How do novices develop such mental models? Most often, they simply resort to adding print statements everywhere in their code to surface the values of certain variables at particular points in execution. Single-step debuggers and automated program visualizers such as my Python Tutor tool have made it easier to inspect selected run-time values without using print statements. Live programming environments continually run the user's code and display usually the most recent value for each executed line, expression, or active object. However, all of these tools only let programmers see a thin slice of run-time state—only a few values at only one execution point at a time. To challenge this status quo, we thought of a weird idea: What if a tool could display all of the program's values all the time?

To explore this idea, we built Omnicode (“Omniscient Code”), a live programming environment that continually runs the user's code and displays the entire history of all numerical program values at all execution steps all the time.

Here's how Omnicode works (click image to enlarge):

  1. Omnicode is meant to be used in a Python-based programming course or MOOC. The user starts with a programming problem statement, a set of test cases to try to pass, and a code editor seeded with skeleton starter code.
  2. As the user is coding, Omnicode continuously runs their code, reports test status and error messages in the left pane, and displays the values of all program variables in the right pane. There is no “Run” button; everything updates live.
  3. The run-time value visualization is a matrix of scatterplots. Each numerical value is plotted both relative to all execution steps (to show the entire history of how it changed over time) as well as relative to all other values (to show dyadic two-variable correlations).
  4. Omnicode automatically derives relevant numerical values for non-numeric types such as the lengths of lists and strings. The user can also write arbitrary Python expressions to visualize on the scatterplots, such as sum(x) or pow(x,2) for the sum of list elements or x2, respectively.
  5. To counteract visual overload, Omnicode implements bidirectional brushing-and-linking: The user can select a region of source code (e.g., a variable or set of lines) and the scatterplot matrix gets filtered to include only data points relevant to the user’s code selection. Conversely, the user can select any region of any scatterplot, and all other scatterplots as well as the source code selection get updated to include only data that is relevant to the user's selection.
  6. Finally, to probe run-time state in more detail, the user can select a line of code and see a pop-up tooltip that shows a complete heap visualization of all data structures present when that line executes (powered by my Python Tutor tool). Whereas the scatterplot matrix shows only numbers, this inline visualization shows values of all Python data types. This feature allows the user to zoom in to get a detailed view of specific execution steps.

When we tested Omnicode on students and teaching assistants, they were excited about using it in two main ways.

First, as expected, they liked using Omnicode's visualizations to help them form mental models of how their code was behaving. Some even made serendipitous discoveries about their code's behavior by glancing at the always-on visualizations and embarked on impromptu explorations that they wouldn't have done within a normal IDE.

What we found more interesting, though, was how participants wanted to use Omnicode as a communication aid to facilitate meaningful dialogue about how their code executed. For instance, teaching assistants could see themselves using it as a pedagogical aid to visually explain their live code demos in class, even if they don't necessarily use it as a full-blown IDE to write significant amounts of code. Omnicode could also help novices perform self-explanations to better verbalize their mental models and discuss misconceptions with tutors.

In sum, we're excited by the potential of Omnicode to give students and instructors a concrete visual substrate to discuss the details of code execution, since it lets all run-time values be laid out on-screen at once rather than hidden behind messy print statements or arcane debugger commands.


Read the full paper for details:

Omnicode: A Novice-Oriented Live Programming Environment with Always-On Run-Time Value Visualizations. Hyeonsu Kang and Philip J. Guo. ACM Symposium on User Interface Software and Technology (UIST), 2017.
Visualizations of run-time program state help novices form proper mental models and debug their code. We push this technique to the extreme by posing the following question: What if a live programming environment for an imperative language always displays the entire history of all run-time values for all program variables all the time? To explore this question, we built a prototype live IDE called Omnicode ("Omniscient Code") that continually runs the user's Python code and uses a scatterplot matrix to visualize the entire history of all of its numerical values, along with meaningful numbers derived from other data types. To filter the visualizations and hone in on specific points of interest, the user can brush and link over the scatterplots or select portions of code. They can also zoom in to view detailed stack and heap visualizations at each execution step. An exploratory study on 10 novice programmers discovered that they found Omnicode to be useful for debugging, forming mental models, explaining their code to others, and discovering moments of serendipity that would not have been likely within an ordinary IDE.
@inproceedings{KangUIST2017,
 author = {Kang, Hyeonsu and Guo, Philip J.},
 title = {Omnicode: A Novice-Oriented Live Programming Environment with Always-On Run-Time Value Visualizations},
 booktitle = {Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology},
 series = {UIST '17},
 year = {2017},
 publisher = {ACM},
 address = {New York, NY, USA},
}
Created: 2017-10-03
Last modified: 2017-10-03
Related pages tagged as human-computer interaction:
Related pages tagged as programming:
Related pages tagged as computing education: