The C++ Tracing Tutor: Visualizing Computer Program Behavior for Beginning Programming Courses Rika Yoshii Alastair Milne Computer Science Department California State University, San Marcos, USA

Background We developed a Web-delivered tutoring system for students in beginning programming courses. The Tutor is a Java applet that has been used at California State University, San Marcos in CS111 and CS211 classes. Developed under MacOS 8.5 and MacOS 9 with Metrowerks CodeWarrior (JDK1.1) Tested under MacOS 8.5 and 9, Windows 2000, and IBM’s AIX, a version of UNIX.

Common Student Problem –They cannot relate their programs to the sequence of machine actions they cause. –They were unable to analyze why a program works or does not, seeing no connection to the resulting actions by the machine. –The same students needed repeated explanations with examples.

Main Features  Visual representations of programs: the Tutor provides a live media visualization of the actions taken by the machine in executing a running program.  Conversational tutoring: the Tutor provides frequent interactions with free-form student answers.  Adaptive learning: the Tutor uses student answers to provide appropriate hints and to select the next exercise.  Mastery learning: the Tutor does not let the student move onto a different program example until the student shows no difficulty understanding the current one. Uses example programs written in the programming language C++, but its concepts can be used for any high-level programming language.

Two Forms Free Exploration – student controlled Conversational Tutor – system controlled –Asks the student to predict the program behavior Five different types of example programs that are often seen in CS111 with for-loops and while- loops. Permits adding more example programs without making major changes to the system (Each example program is a syntax tree of objects.)

Visual Model of Execution Shows the student which statement is executing at a time (color coded). Simulated computer memory shows the variable contents. Simulated keyboard shows input values typed by an imaginary user. Simulated display shows the outputs.

Tutoring Sequence If a wrong answer is given, –the tutor causes the example program to perform a partial execution at a measured pace before posing the question again. If the student is persistently unable to predict correctly, –the entire sequence of execution may be done, so that the answer will be displayed for the student. If the student has shown little or no difficulty in answering questions correctly on up to four different aspects of the program, –move onto another program with a slightly different or more challenging concept. If the student has needed a significant number of retries to work out correct answers, or has missed them altogether, –stay with the current program, but replace various constant values it contains with randomly-selected new values

Student Reactions Questionnaire to several advanced CS majors and three sections (approx 60 students) of the CS111 class to discover any problems with the user interface. –to improve the color scheme, to increase the size of the instruction window, improve the labels on the buttons, and to make it clear that the simulated keyboard and input display are for the simulated user.

Student Performance Using 5 sections of CS111, we collected data on any changes in the students’ programming as a result of using the Tutor. Preliminary analyses (of 4 sections) –Average improvement of 15% in points between the pre and post-tests after using the system for only two hours. –Pre-test : for-loop to obtain highest, total and average. –Post-test: while-loop with sentinel value to obtain highest, total and average. –Given 1 hour each. –No human instruction on the related materials between the tests.

Some Ideas in Building Applets A small amount of dynamic HTML in which Javascript code detects the hosting browser and operating system, and dynamically writes the applet-inclusion element that works for that form. If the browser and operating system are not recognized, the Javascript assumes they conform to HTML 4.0 standards. In order to minimize the chance of the Javascript itself being unusable to some browsers, its version and operations were kept as simple as possible.

Future Tasks Analyzing data on how programming errors change as the result of using the Tutor. Analyzing data on how student performance changes if only the Exploration module is used. Repeat the tests with more students and have a control group. Create additional tracks for more advanced constructs that are taught in introductory programming courses (e.g. arrays, functions, and switch). Create a separate remedial track for those students having problems with specific constructs. Provide an ability to add or change example programs easily by creating a translator that reads a text file containing an example program and builds a Java object representation of it. Current questions require us to look for simple values and expressions. As we include questions requiring more complex textual answers, we will need to incorporate a more powerful answer analyzer.