Philip Guo (Phil Guo, Philip J. Guo, Philip Jia Guo, pgbovine)

Toward a Domain-Specific Visual Discussion Forum for Learning Computer Programming: An Empirical Study of a Popular MOOC Forum

research paper summary
Toward a Domain-Specific Visual Discussion Forum for Learning Computer Programming: An Empirical Study of a Popular MOOC Forum. Joyce Zhu, Jeremy Warner, Mitchell Gordon, Jeffery White, Renan Zanelatto, Philip J. Guo. IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), 2015.
Online discussion forums are one of the most ubiquitous kinds of resources for people who are learning computer programming. However, their user interface -- a hierarchy of textual threads -- has not changed much in the past four decades. We argue that generic forum interfaces are cumbersome for learning programming and that there is a need for a domain-specific visual discussion forum for programming. We support this argument with an empirical study of all 5,377 forum threads in Introduction to Computer Science and Programming Using Python, a popular edX MOOC. Specifically, we investigated how forum participants were hampered by its text-based format. Most notably, people often wanted to discuss questions about dynamic execution state -- what happens "under the hood" as the computer runs code. We propose that a better forum for learning programming should be visual and domain-specific, integrating automatically-generated visualizations of execution state and enabling inline annotations of source code and output.
@inproceedings{ZhuVLHCC2015csforums,
  author={Zhu, Joyce and Warner, Jeremy and Gordon, Mitchell and White, Jeffery and Zanelatto, Renan and Guo, Philip J.},
  title={Toward a Domain-Specific Visual Discussion Forum for Learning Computer Programming: An Empirical Study of a Popular MOOC Forum},
  booktitle = {Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)},
  series = {VL/HCC '15},
  year={2015},
  pages={101-109},
  doi={10.1109/VLHCC.2015.7357193},
  month={Oct}
}

Online discussion forums are everywhere, but their user interfaces haven't changed much since their early incarnations over 40 years ago as Usenet newsgroups and The WELL. For instance, here's a forum from a computer programming MOOC showing the usual hierarchy of text-based topics, threads, and replies:

While this format is fine for ordinary everyday questions (e.g., “What did you like about the new Star Wars movie?”), we suspect that it's really cumbersome for discussing complex technical topics such as computer programming.

To investigate this hunch, we manually read all 5,377 discussion forum threads in the Spring 2015 offering of the edX MOOC Introduction to Computer Science and Programming Using Python. Here's what we found and what the implications are for rethinking user interfaces for programming-related forums:

Finding 1: The majority of forum threads in this MOOC (~60%) mentioned code snippets, outputs and errors resulting from execution, run-time execution state, or complaints about the autograder tool that grades assignments based on test cases.

Implication: A forum optimized for learning programming must support inline anchored discussions around the above topics, thus eliminating the need to copy-and-paste code and outputs into disconnected threads. One possible solution is to use pop-up tooltips to anchor discussions into Python Tutor visualizations, shown as blue bubbles in this UI mockup (click to enlarge):

This way, students can directly hold discussions around specific components of run-time execution state.

Finding 2: Existing forums have no built-in way of visualizing run-time execution state, so students must resort to roundabout textual explanations or manually-drawn diagrams of such state. For instance, this student took a photo of a hand-drawn diagram to clarify their post:

Implication: A forum should integrate tightly with a program visualization tool (such as the Python Tutor mockup above), thus eliminating the need for manually-drawn diagrams.

Finding 3: Existing forums are not designed for holding rich conversations around source code; they treat code simply as plain text with optional formatting. As an extreme example, when students forgot to put their code in specially-formatted blocks, it often looked like a mess:

Implication: A programming forum should treat code not simply as blocks of plain text, but rather as first-class objects that can be annotated, versioned, and semantically linked to code posted by other students.

Finding 4: Existing forums are not linked with a code execution engine, so they can't capture discussions around program outputs and errors.

Implication: A forum should also integrate with an IDE so that students can execute code, see outputs and errors, and discuss them all within the IDE.

Finding 5: The autograder tool is a central component of programming courses, but how it works is often unclear to students. Thus, a fair amount of discussion threads involve complaints about this tool.

Implication: A program visualization tool such as Python Tutor should be extended to visualize how the autograder works, in order to provide more transparency to students. In addition, integrating the forum with these visualizations can allow students to more easily discuss grading-related issues.

In sum, this study surfaced some shortcomings of using general-purpose forum software for computer programming topics. Our design implications can help future MOOCs, open-source projects, and Q&A sites such as Stack Overflow improve their discussion interfaces to be more helpful for novice programmers.


Read the full paper for details:

Toward a Domain-Specific Visual Discussion Forum for Learning Computer Programming: An Empirical Study of a Popular MOOC Forum. Joyce Zhu, Jeremy Warner, Mitchell Gordon, Jeffery White, Renan Zanelatto, Philip J. Guo. IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), 2015.
Online discussion forums are one of the most ubiquitous kinds of resources for people who are learning computer programming. However, their user interface -- a hierarchy of textual threads -- has not changed much in the past four decades. We argue that generic forum interfaces are cumbersome for learning programming and that there is a need for a domain-specific visual discussion forum for programming. We support this argument with an empirical study of all 5,377 forum threads in Introduction to Computer Science and Programming Using Python, a popular edX MOOC. Specifically, we investigated how forum participants were hampered by its text-based format. Most notably, people often wanted to discuss questions about dynamic execution state -- what happens "under the hood" as the computer runs code. We propose that a better forum for learning programming should be visual and domain-specific, integrating automatically-generated visualizations of execution state and enabling inline annotations of source code and output.
@inproceedings{ZhuVLHCC2015csforums,
  author={Zhu, Joyce and Warner, Jeremy and Gordon, Mitchell and White, Jeffery and Zanelatto, Renan and Guo, Philip J.},
  title={Toward a Domain-Specific Visual Discussion Forum for Learning Computer Programming: An Empirical Study of a Popular MOOC Forum},
  booktitle = {Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)},
  series = {VL/HCC '15},
  year={2015},
  pages={101-109},
  doi={10.1109/VLHCC.2015.7357193},
  month={Oct}
}
Created: 2015-10-18
Last modified: 2017-10-23
Related pages tagged as computing education:
Related pages tagged as human-computer interaction: