1. Methods for Human- Computer Interactions (HCI) Research Dr. Xiangyu Wang Design Computing x.wang@arch.usyd.edu.au Acknowledgement to Sasha Giacoppo and Kerstin, Severinson Eklundh, and Ann Lantz for lecture notes

2. The purpose of the lecture  To analyze and discuss HCI as a science  To review different research methods, and how they are used in HCI  To learn how to plan and perform a scientific investigation in HCI at a basic level

3. Facts about HCI  HCI is a young discipline (about 30 years)  Has developed from computer science, with influences from cognitive psychology  Also influenced by sociology, anthropology, communication, design, ergonomics, etc.

4. HCI as a ”user science”  All HCI research includes studies of the use of technology  Empirical study methods are essential  HCI is also closely related to the design and development of technology  Important to discuss the relationship between research and development

5. HCI  HCI has a wide variety of professional and academic arenas, including:  Information Technology & Computing  Design & Development  Business  Entertainment  Education  Library Sciences

6. HCI  More common areas of research related to design computing areas:  Website design and construction  Consumer product design and development  Data Management and Information Visualization  Ergonomics and Industrial Design  Online Communities and Collaborative Systems  Entertainment and Gaming

7. How to do HCI research?  Read, research, and understand the numerous theories and domains that investigate HCI related issues.  Select the domains of HCI that specifically support your objectives. Read and research the theories and models specific to those domains you selected.  Then, review the methodologies available for your investigation.  Choose the appropriate methodologies, based on the desired breadth and depth of your investigation  Use your investigations to build theories and/or models.

8. Methods for Data Collection  Measurement  How can variations in a phenomenon be measured?  Subjective measures  Ratings, question-answer studies  Objective measures  Time measurements, or process logs  Validity: do you measure what you intended measure?  Reliability: do different measurements give the same result?

9. Methods for Data Collection  Procedure for data collection and analysis  Consider validity and reliability  Will the procedure answer the correct question? Will it do so in a reliable way?  Systematic way of collecting information  Always be consistent; the same level of detail everywhere  Document the procedure systematically

10. HCI Research Methods  HCI research methods can be grouped into the following categories:  Development Methods: Task Analyses  Survey Methods: Surveys & Questionnaires  Controlled Experiments  Ethnographic Methods  Logging & Automated Metrics

11. Development Methods: Task Analyses  Usability engineering (engineering, development, and usability methodologies)  A task analysis is not a single analysis, but rather a collection of many different analyses.

12. Development Methods: Task Analyses  According to Hackos (1998), a user needs assessment and task analysis seeks to understand:  What user’s goals are; What they are trying to achieve  What users actually do to achieve those goals  What personal, social, and cultural characteristics the users bring to the tasks  How users are influenced by their physical environment  How users’ previous knowledge and experience influence how they think about their work and the workflow they follow to perform their tasks  What users value most that will make a new interface be a delight for them (i.e. speed? Accuracy? Error recovery? Human contact? Fun?)

13. Techniques of Task Analysis  Workflow analysis (business process analysis)  Performed to understand how a user, or a group of users, accomplishes a particular process.  A horizontal picture of how work moves across and between people. Workflow analysis is important because it allows investigators to identify which users are doing which tasks in a process.  Importance to HCI: it does support HCI researchers in their understanding of users. More importantly, workflow analysis provides HCI researchers with important information of user behavior in context. User 1User 2User 3 Sets of Activities A Sets of Activities B Sets of Activities C Workflow

14. Techniques of Task Analysis  Job analysis  Performed to understand the activities of a single user within a process.  A vertical picture of all the activities that flow through a single user. A job analysis should involve direct observation of a person over time, or at the very least, a logging of daily activities by a person.  Importance to HCI: Job analysis can capture meaningful, quantitative data about how and where users spend their time. Information like frequency, time to complete, difficulty, etc. can all greatly help researchers designers and developers when it comes to developing, improving, discovering, etc. interfaces and products for users. User 1User 2User 3 Activity a Sets of Activities B Sets of Activities C Activity b …

15. Techniques of Task Analysis  Task hierarchies  A task hierarchy illustrates the tasks and subtasks of a process or activity. It seeks to deconstruct and decompose tasks into its smallest components  Importance to HCI: It provides researchers with a picture of how many levels of tasks are involved in a process. The researcher can then decide which levels and how many levels of a process need to be investigated. User 1 /Activity a User 2User 3 Task a Sets of Activities B Sets of Activities C Task b …

16. Techniques of Task Analysis  Procedural analysis  One specific task that is divided into the steps and decisions that a user goes through in doing that task. It illustrates how users carry out their tasks with the tools currently available to them.  Importance to HCI: Procedural analysis can create an idealized picture of the process that users follow, or an actual picture of the process that users follow. From this, areas that need to be improved, or where users make a lot of errors, can be identified, or investigated further. Task 1Task 2Task 3 User 1/Activity a

17. Advantages/Disadvantages of Task Analysis  Advantages: Task analysis is a complex battery of techniques intended to provide a researcher with a complete understanding of users/tasks.  Disadvantages: A task analysis can be an extremely time-and resource-consuming affair. The more information you desire to collect, the longer you will have to burden users with observation and data collection.

18. Development Methods: Task Analyses  User and task analysis is perfectly suited for development-oriented researchers and designers. The results of user and task analyses can be used in several phases of the development process (Tucker, 1997):  Development of requirements  User Interface Design & Evaluation  Follow-up After Installation/Use of System/Product

19. Techniques for collecting data  Observations  Interviews (visiting, telephone, indirect)  Questionnaire  Diary  …..

20. Questionnaires  Before conducting a study -practicalities  Time  Cost  Range  Questions (type, wording, order)  Demands reflection  Instructions  Pictures, anonymous cards  Language, knowledge

21. Checklist for constructing a questionnaire  Be concrete  Think about effects due to the order of posing your questions (context)  Ambiguity  Assumptions  Careful with yes and no questions

22. Question type  Open questions  List  Category  Ranking  Scale  Quantity  Table

23. Appearance and layout  Word processed  Clear instructions  Spacing between questions  Keep response boxes in line (left/right)  Guide the respondent the right way  Promise anonymity and confidentiality

24. Administering the questionnaire  Electronic, snail mail or face-to-face  Self-addressed envelope  Instruction, information letter  Anonymous, confidential

25. Piloting the questionnaire  How long did it take you to complete?  Were the instructions clear?  Were any of the questions unclear or ambiguous? If so, will you say which and why?  Did you object to answering any of the questions?  In your opinion, has any major topic been omitted?  Was the layout of the questionnaire clear/attractive?  Any comments?