Report of the CMU Natural Programming Group Brad Myers, Andy Ko, Jeff Stylos, Michael Coblenz, Brian Ellis, Polo Chao Carnegie Mellon University

Previous Years Alice Whyline – debug by asking why questions Slate – Objects of Measurement in a spreadsheet Mica – Uses Google to help find APIs Crystal – Why questions in applications Citrus – Toolkit for structured editors Barista – Editor for code Numerous studies of behaviors

3 This Year Mostly relevant to professional programmers Study of Bug Report Linguistics Jasper, for tracking task relevant information Study of the “Factory Pattern” Information needs of Microsoft developers Preliminary work on Java Whyline feasibility (15 papers submitted or published)

4 Study of Bug Report Linguistics Searched for patterns in report titles from 180,000 open source bug reports Used a small subset of English A large set of proper names Contained subject of failure, quality attribute, problem with quality and context of problem People identified problems by referring to output, user actions, and hardware

5 Jasper Tool for Working Sets CSD MS thesis of Michael Coblenz Jasper = Java Aid with Sets of Pertinent Elements for Recall Allow programmers to grab arbitrary fragments of code to represent working sets ‣ Also notes, documentation, and other meta- information about code Allow programmers to view fragments in one place, one screen Use for navigation, reminders, bookmarks, collaboration One working set for each task

6 Preliminary Implementation Eclipse plug-in ‣ Enormous difficulties and overhead in implementation

7 Study of APIs Work of PhD student Jeff Stylos How do design patterns affect API usability? ‣ While at Microsoft: constructors ‣ Summer: factory pattern All programming systems (end-user or professional) have APIs Measures: learnability, errors, preferences Expert and novice programmers Submitted to ICSE’07

8 Constructor Study at Microsoft Internal debate about whether to provide “default” (null) constructors in MS APIs ‣ MS devs: null constructors can cause errors Study of 30 professional programmers Results: ‣ Programmers used constructors differently than expected ‣ Null constructors actually helped (matched expectations, were faster, didn’t cause errors) ‣ True for novices and experts

9 “Factory” Pattern Objects must be created by another class: AbstractFactory f = AbstractFactory.getDefault(); Widget w = f.createWidget(); Used frequently in Java (>61) and.NET (>13) Lab study with 12 expert Java programmers ‣ Five programming and debugging tasks ‣ Within subject and between subject measures Results: ‣ No one expected factories when coding in Eclipse or on paper ‣ Finding and using factories took 2.1 to 5.2 times longer than using constructors, or an extra 6 to 11 minutes per object ‣ All subjects had difficulties using factories in APIs Recommendations: ‣ Use constructors or other patterns with same architectural advantages

10 Information Needs of MS Developers Andy’s summer MSR internship Observed 17 developers work on writing, understanding, debugging, designing Took notes about: ‣ The information looked for ‣ Where information was found ‣ What prevented it from being found Classified types, sources, outcomes

11 Information Needs of MS Developers Variation in task structure, interruptions and getting blocked Seven most deferred types of info ‣ What has my team been doing? ‣ How have my resources changed? ‣ In what situations does this failure occur? ‣ What could have caused this behavior? ‣ What actually caused this behavior? ‣ Why was this code written this way? ‣ What is this code supposed to do?

The Whyline Andy Ko’s thesis project ‣ Officially proposed last May Generalize the Whyline to general purpose imperative languages, larger programs Questions: ‣ What is the “output” at a given point in time that users will want to ask about? ‣ What information must we record to effectively answer users’ questions? ‣ How can users ask Why Didn’t questions about a lack of output? Develop infrastructure for instrumentation, tracing, and relating code to trace Instrumentation almost done

13 Next Year Finish and evaluate the Java Whyline More studies of API design decisions Extend work on Jasper