1. Copyright © 2005 – Brad A. Myers End Users in End-User Software Engineering: Where HCI Cross Cuts SE ̶ First Workshop on End-User Software Engineering (WEUSE) Brad A. Myers Human Computer Interaction Institute School of Computer Science Carnegie Mellon University Brad A. Myers Human Computer Interaction Institute School of Computer Science Carnegie Mellon University

2. 2May 21, 20052Brad A. Myers, CMU Human-Computer Interaction HCI (at least at CMU) = CS Social Sciences Designers CS contributes New tools and techniques Social Sciences contribute Evaluation & processes Designers contribute Better appearance and interaction HCI (at least at CMU) = CS Social Sciences Designers CS contributes New tools and techniques Social Sciences contribute Evaluation & processes Designers contribute Better appearance and interaction

3. 3May 21, 20053Brad A. Myers, CMU HCI Especially Important for WEUSE Even more so than “regular” SE All programmers are still Human Still must Interact with Computers But end-user programmers are especially “not like me” Even more so than “regular” SE All programmers are still Human Still must Interact with Computers But end-user programmers are especially “not like me”

4. 4May 21, 20054Brad A. Myers, CMU Why Why not “Just don’t” for EUP? SE attitude? How to let the millions of computer users take full advantage of computers Computers are malleable and customizable But only if you know how Why not “Just don’t” for EUP? SE attitude? How to let the millions of computer users take full advantage of computers Computers are malleable and customizable But only if you know how

5. 5May 21, 20055Brad A. Myers, CMU How Can HCI Methods Be Helpful? What is “known” by the HCI Community that might not be known by SE Community Knowledge about what works and doesn’t Methods for approaching design and evaluation Appropriate statistical methods Techniques for interaction & visualization Toolkits for implementation (nothing on this topic here?) What is “known” by the HCI Community that might not be known by SE Community Knowledge about what works and doesn’t Methods for approaching design and evaluation Appropriate statistical methods Techniques for interaction & visualization Toolkits for implementation (nothing on this topic here?)

6. 6May 21, 20056Brad A. Myers, CMU HCI Issues that Span Many Papers Many of these issues appear in many of this workshop’s papers Topics I noticed: Need to Understand Users New Interaction Techniques and Visualizations for the Systems Evaluating the Resulting Systems Maps to the phases of HCI development Many of these issues appear in many of this workshop’s papers Topics I noticed: Need to Understand Users New Interaction Techniques and Visualizations for the Systems Evaluating the Resulting Systems Maps to the phases of HCI development

7. 7May 21, 20057Brad A. Myers, CMU Need to Understand Users Seen in Scaffidi, Prechelt, Pickard, Segal, Sutcliffe, Ko, … Sub-topics Properties of users Motivation of users Task analysis “Formative” studies To find out what to do (As opposed to “summative” or evaluation studies done after the system is built or prototyped) Seen in Scaffidi, Prechelt, Pickard, Segal, Sutcliffe, Ko, … Sub-topics Properties of users Motivation of users Task analysis “Formative” studies To find out what to do (As opposed to “summative” or evaluation studies done after the system is built or prototyped)

8. 8May 21, 20058Brad A. Myers, CMU Properties of Users HCI principles: “Know the user” “The user is not like me” “Don’t blame the users” Lots of different kinds of users, and lots of kinds of “programming” Can’t generalize from what is appropriate for programming for a bank to what is needed for home or scientific use Professional programmers  Professional End- User Developers  “Casual” EUDs Judith Segal’s review of scientists and financial consultants Also, users of the abaXX system HCI principles: “Know the user” “The user is not like me” “Don’t blame the users” Lots of different kinds of users, and lots of kinds of “programming” Can’t generalize from what is appropriate for programming for a bank to what is needed for home or scientific use Professional programmers  Professional End- User Developers  “Casual” EUDs Judith Segal’s review of scientists and financial consultants Also, users of the abaXX system

9. 9May 21, 20059Brad A. Myers, CMU Properties of Users by Survey Scaffidi, Shaw and Myers Don’t use the 55 million number anymore! Analyzed Boehm’s paper, BLS and other data Real numbers (2001): 72 million users, 45 million spreadsheet users, 11 million “do programming,” 3 million professional programmers What do people use in their EUP environments Try to generalize to what they know in terms of abstractions Variables, functions, data structures Scaffidi, Shaw and Myers Don’t use the 55 million number anymore! Analyzed Boehm’s paper, BLS and other data Real numbers (2001): 72 million users, 45 million spreadsheet users, 11 million “do programming,” 3 million professional programmers What do people use in their EUP environments Try to generalize to what they know in terms of abstractions Variables, functions, data structures

10. 10May 21, 200510Brad A. Myers, CMU Motivation of Users Blackwell’s model of “Attention Investment” A. F. Blackwell. “First steps in programming: a rationale for attention investment models,” IEEE HCC-VL’02, Arlington, VA. pp.2-10 Costs: learning time & actual programming time Time away from the “real work” Risks: won’t work & be a waste of time Benefits: future savings if task done again Also utility Is it really useful? Segal, Sutcliff  sometimes! Attention Investment [Joseph Ruthruff & Margaret Burnett] “If we built it, will they come?” How make benefits more real and more apparent? How make risks and costs less? Blackwell’s model of “Attention Investment” A. F. Blackwell. “First steps in programming: a rationale for attention investment models,” IEEE HCC-VL’02, Arlington, VA. pp.2-10 Costs: learning time & actual programming time Time away from the “real work” Risks: won’t work & be a waste of time Benefits: future savings if task done again Also utility Is it really useful? Segal, Sutcliff  sometimes! Attention Investment [Joseph Ruthruff & Margaret Burnett] “If we built it, will they come?” How make benefits more real and more apparent? How make risks and costs less?

11. 11May 21, 200511Brad A. Myers, CMU Classic “Gentle Slope Systems” Brad A. Myers, David Canfield Smith and Bruce Horn. “Report of the `End-User Programming’ Working Group,” Languages for Developing User Interfaces, Boston, MA, Jones and Bartlett. 1992. pp. 343-366.

12. 12May 21, 200512Brad A. Myers, CMU Task Analysis Find out what users are really doing or need to do Might ask users (surveys, questionnaires) or introspect, but: Users are not like us Users tell what they think they want, not what they need Surveys, etc. get at marketing data, not usage Self reports not valid Dramatic bad events are the most salient People over-estimate time for mentally challenging activities Under-estimate time on routine tasks So need to see what people actually do Find out what users are really doing or need to do Might ask users (surveys, questionnaires) or introspect, but: Users are not like us Users tell what they think they want, not what they need Surveys, etc. get at marketing data, not usage Self reports not valid Dramatic bad events are the most salient People over-estimate time for mentally challenging activities Under-estimate time on routine tasks So need to see what people actually do

13. 13May 21, 200513Brad A. Myers, CMU Evidence from abaXX Lutz Prechelt and Daniel Hutzel Found that many of the EUP facilities were not appropriate Not useful, not allowed Eventually, useful for understanding rather than creating Lutz Prechelt and Daniel Hutzel Found that many of the EUP facilities were not appropriate Not useful, not allowed Eventually, useful for understanding rather than creating

14. 14May 21, 200514Brad A. Myers, CMU “Contextual Inquiry” In HCI at CMU, we teach the “Contextual Inquiry” technique Most understandable, teachable and usable method for task analysis Find user’s real tasks But much more: Context!!! Cultural influences (corporate, social, etc.) Might have identified some of abaXX’s issues before design Reference: H. Beyer and K. Holtzblatt. Contextual Design: Defining Custom-Centered Systems. San Francisco, CA, Morgan Kaufmann Publishers, Inc. 1998. http://www.incent.com/cd/cdhow.html In HCI at CMU, we teach the “Contextual Inquiry” technique Most understandable, teachable and usable method for task analysis Find user’s real tasks But much more: Context!!! Cultural influences (corporate, social, etc.) Might have identified some of abaXX’s issues before design Reference: H. Beyer and K. Holtzblatt. Contextual Design: Defining Custom-Centered Systems. San Francisco, CA, Morgan Kaufmann Publishers, Inc. 1998. http://www.incent.com/cd/cdhow.html

15. 15May 21, 200515Brad A. Myers, CMU How Contextual Inquiry Works Interpretive field research method Depends on conversations with users in the context of their work Recommends “direct observation” when possible When not possible Cued recall of past experience, or Recreation of related experience Used to define requirements, plans and designs. Develop diagrams to communicate results and show breakdowns Data flow Sequence of steps performed Cultural influences Physical environment of work Artifacts used and created Interpretive field research method Depends on conversations with users in the context of their work Recommends “direct observation” when possible When not possible Cued recall of past experience, or Recreation of related experience Used to define requirements, plans and designs. Develop diagrams to communicate results and show breakdowns Data flow Sequence of steps performed Cultural influences Physical environment of work Artifacts used and created

16. 16May 21, 200516Brad A. Myers, CMU Why Context ? Design complete work process Fits into “fabric” of entire operations Not just “point solutions” to specific problems Integration! Consistency, effectiveness, efficiency, coherent Design from data Not just opinions, negotiation Not just a list of features Design complete work process Fits into “fabric” of entire operations Not just “point solutions” to specific problems Integration! Consistency, effectiveness, efficiency, coherent Design from data Not just opinions, negotiation Not just a list of features

17. 17May 21, 200517Brad A. Myers, CMU Examples Andy Ko and our uses of CI Revealed need for why not questions and support for false assumptions during debugging  WhyLine Revealed patterns of activity during maintenance  this year’s ICSE paper Revealed how people edit programs  future work on structure editors Earlier work: John Pane’s PhD on “Hands” How people think about algorithms, etc. E.g., “Men and women raise your hands”; “Move all the chairs into a circle.” Andy Ko and our uses of CI Revealed need for why not questions and support for false assumptions during debugging  WhyLine Revealed patterns of activity during maintenance  this year’s ICSE paper Revealed how people edit programs  future work on structure editors Earlier work: John Pane’s PhD on “Hands” How people think about algorithms, etc. E.g., “Men and women raise your hands”; “Move all the chairs into a circle.”

18. 18May 21, 200518Brad A. Myers, CMU New Interaction Techniques and Visualizations How can users control the system? How convey information to users? Many innovations in EUP areas: Earlier work: PBD techniques; Rich McDaniel’s “Do Something” button, “hint highlighting”, “deck of cards” Andy Ko’s why button and “WhyLine” visualization New interaction techniques planned for text editing of code WYSIWYT & Markus Clermont: highlighting, “x? ”, etc. HCI can help with design so attractive, effective, professional-looking Color choice, icon design, layout, etc. How can users control the system? How convey information to users? Many innovations in EUP areas: Earlier work: PBD techniques; Rich McDaniel’s “Do Something” button, “hint highlighting”, “deck of cards” Andy Ko’s why button and “WhyLine” visualization New interaction techniques planned for text editing of code WYSIWYT & Markus Clermont: highlighting, “x? ”, etc. HCI can help with design so attractive, effective, professional-looking Color choice, icon design, layout, etc.

19. 19May 21, 200519Brad A. Myers, CMU Design by Programmer vs. Designer

20. 20May 21, 200520Brad A. Myers, CMU Evaluating the Resulting System Many HCI methods for evaluation Heuristic Evaluation Method Usability Testing Lab or Field studies Cognitive Walkthrough Analytic methods Keystroke model, GOMS model Questionnaires, Interviews, Focus Groups, Logging, Feedback Effectiveness in real deployment Statistics methods for proper evaluation Will be covered more in Gregg’s section Many HCI methods for evaluation Heuristic Evaluation Method Usability Testing Lab or Field studies Cognitive Walkthrough Analytic methods Keystroke model, GOMS model Questionnaires, Interviews, Focus Groups, Logging, Feedback Effectiveness in real deployment Statistics methods for proper evaluation Will be covered more in Gregg’s section

21. 21May 21, 200521Brad A. Myers, CMU Many Insights Revealed in Studies Studies not only to show improvement percentages for the papers! Reveals interesting new research problems and designs E.g., Ruthruff: misconceptions: end-user programmers make errors when testing (up to 25%), and affects 60% of success rates Ko: WhyLine: people didn’t use reverse execution, instead used events; originally had why “did/didn’t” and “is/isn’t” but confusing Might not have thought about these issues w/o information from users Studies not only to show improvement percentages for the papers! Reveals interesting new research problems and designs E.g., Ruthruff: misconceptions: end-user programmers make errors when testing (up to 25%), and affects 60% of success rates Ko: WhyLine: people didn’t use reverse execution, instead used events; originally had why “did/didn’t” and “is/isn’t” but confusing Might not have thought about these issues w/o information from users

22. 22May 21, 200522Brad A. Myers, CMU HCI + SE Real opportunities for innovation Renewed interest from ICSE, industry and community in general Award winning ideas possible! Real opportunities for innovation Renewed interest from ICSE, industry and community in general Award winning ideas possible!