Towards understanding programs through wear-based filtering Robert DeLine Amir Khella Mary Czerwinski George Robertson Microsoft Corporation SoftVis 2005

Developers often change unfamiliar code Today’s business makes it unavoidable. Team membership is always changing Separate teams create features and fixes Security teams consult across products Teams consume outsourced products How could today’s tools be better for this task? We studied programmers to find usability issues We suggest an approach for improving tools

A pilot study and a hunch We ran a pilot study and watched participants... Take lots of fruitless navigation steps Navigate to sort through the results of searches Get frequently lost in the program text Hunch: one can learn the important parts of the code by watching code navigation Hypothesis: navigation frequency correlates with importance ratings

Formative, observational study Participants run singly in two hour sessions 10 min code exploration, 20 min per task We logged navigation and videoed Tasks followed by quiz and ratings questionnaire Dependent measures Code traversal paths Subjective importance ratings (1 to 5) Plus, task completion and quiz scores

Study participants 7 experienced programmers All male, average age 36 Average 18 years programming All work outside Microsoft We screened for… Working in teams, on long-term projects Debugging or modifying others’ code Familiarity with Visual Studio, GUI programming

Participants modified a Tetris game Code selected for familiarity, size, and high rating Two comprehension tasks, two modification tasks 1 Which method chooses the next game piece? 2 Which method causes pieces to fall faster? 3 Add the missing square game piece. 4 Add a command to make the piece fall immediately to the bottom.

Importance ratings do correlate to navigation frequency Pearson product moment correlation, r=0.79, p<0.01

Study results Poor completion of modification tasks 7 / 7 completed tasks 1 and 2 1 / 7 completed task 3 1 / 7 completed task 4 (different participant) Participants used different learning styles All started by exploring control flow 2 / 7 used top-down, hypothesis-driven approach 5 / 7 used bottom-up approach and visited more code

Usability problems Participants wanted more overview information Architectural diagrams, e.g. UML class diagrams Descriptions of the relationships among components Short summaries of important algorithmic steps Participants got lost in the program text Had difficulty managing many open documents Needed extra navigation to explore search results Wanted help in tracking exploration (bread crumbs) Participants overused short-term memory Difficulty recognizing previously visited code Annoyance when they couldn’t use spatial memory

How can we make the task easier? Let’s use navigation data for social filtering The more often code is visited, the greater its importance for a newcomer. The more often parts of code are visited in succession, the more related they are. Team Tracks extends Visual Studio Filters file & class browsers to most visited items Recommends related code based on context

Demo of Team Tracks

Conceptual design not yet implemented UML class diagram with navigation heat map

Closest related work Read wear/edit wear Hill, Hollan, Wroblewski, McCandless ’92 Introduced idea of tracking document usage Focus on team awareness Project Watcher Schneider, Gutwin, Penner, Paquette ’04 Applied read wear/edit ware idea to software artifacts Focus on team awareness ROSE Zimmermann, Weißgerber, Diehl, Zeller ’04 Find change patterns in source history Focus on recommending related code Mylar Kersten, Murphy ’05 Track file-level navigation per individual user Focus on managing user’s working set

On-going work Team Tracks evaluation submitted to VL/HCC ’05 1 Lab study with same setup as formative study 2 Longitudinal study with compiler team Further research Compare navigation data with source history Scope navigation data by person, by time Help newcomers learn process, not just code research.microsoft.com/hip Human Interactions in Programming