1. Part 1-Intro; Part 2- Req; Part 3- Design  Chapter 20 Why evaluate the usability of user interface designs?  Chapter 21 Deciding on what you need to evaluate: the strategy  Chapter 22 Planning who, what, when, and where

2. Jakob Nielsen's set of heuristics  Does the Interface Meet the Usability Requirements? ◦ Effective ◦ Efficient ◦ Engaging ◦ Error tolerant ◦ Easy to learn  Exploring Other Concerns in Evaluations ◦ Why users are unable to complete tasks easily. ◦ Is the UI developed for all levels of users? ◦ Are all design features acceptable to users?  Visibility of system status  Match between system and the real world  User control and freedom  Consistency and standards  Error prevention  Recognition rather than recall  Flexibility and efficiency of use  Aesthetic and minimalist design  Help users recognize, diagnose, and recover from errors  Help and documentation 2

3.  Expert Review  Cognitive Walkthrough  Usability evaluation ◦ Participant ◦ Observer (Logger) ◦ Facilitator (Briefer/debriefer) ◦ Evaluator ◦ Evaluation Data  Raw Data  Qualified Data  Quantified Data 3

4. ◦ Our Running Example: Global Warming  The Users for S103  Users’ Tasks and the Global Warming UI  CD-ROM based  The Domain for the Global Warming UI  Paper-based and CD-ROM materials  The Environment for the Global Warming UI  Home-study, or work ◦ Description of the Global Warming User Interface 4

5. 5

6.  Usability concerns ◦ evidence of visibility, affordance, and feedback  True Users  Task Scenarios with setting 6

7. 7

8. 8

9. ◦ The Process of Usability Evaluation Is Iterative ◦ Techniques for Usability Evaluations  User Observations  Inspections of the User Interface  Conform to usability standards?  Other Evaluation Techniques  Variations of user observation or inspection 9

10. 10

11. ◦ Welcome participant, ask a couple of questions to determine participant’s attitude and current half filled/empty mood ◦ Explain purpose  the system has problems that we have not been able to uncover. ◦ Make participant comfortable. Assure the participant they can stop at any time ◦ Rehearse the “think out loud” with the participant ◦ Give participant the task scenarios with setting to complete tasks  you observe and record in another location – you must give the participant privacy & room ◦ Following completion of tasks, ask for participant’s views & to complete post-test questionnaire ◦ Thank participant. 11

12. ◦ Deciding What to Test ◦ Do You Have to Work within Any Constraints? ◦ Writing Up Your Evaluation Strategy for the Global Warming Evaluation 12

14. ◦ What Is the Purpose of This Evaluation?  Does system meet usability requirements/concerns  Qualitative Usability Requirements  Desired features  “The users on an e-shopping site should be able to order an item easily and without assistance.”  “Railway clerks work in extremely noisy environments, so any warning messages to them should be visually distinct and highlighted on the screens.” 14

15.  Quantitative Usability Requirements/Usability Metrics  Explicit measures used: percentages, timings, or numbers are used.  “It should be possible for the users to load understand any page of a web site in 10 seconds using a 56K modem.”  “It should take no more than two minutes for an experienced user (one who has domain knowledge and has undergone the prescribed level of training when the new system is introduced) to enter know how to enter a customer’s details in the hotel’s database & do so with no more than 2 sec hesitation” 15

16.  Prioritizing Usability Requirements and Concerns  The usability requirements most important to the success of the system are given priority.  Assign values to the five dimensions of usability, the Five Es. 16

17.  General Museum Site 17

18. ◦ What Type of Data Do I Want to Collect?  Quantitative data  Numeric content  Qualitative data  Non-numeric content 18

19. ◦ What Am I Evaluating?  Never a finish product without opportunities to improve ◦ What Constraints Do I Have?  Money  Timescales  Availability of usability equipment  Availability of participants and the costs of recruiting them  Availability of evaluators ◦ Documenting the Evaluation Strategy 19

20. 20

21. ◦ Deciding What to Test ◦ Do You Have to Work within Any Constraints? ◦ Writing Up Your Evaluation Strategy for the Global Warming Evaluation 21

22. CHAPTER 22 PLANNING WHO, WHAT, WHEN, AND WHERE

23. Who Is a True User? Users who reflect the different skills, domain knowledge, system experience Key True User Questions – what characteristics must be present to insure a true user Number of Participants to have valid results N=6 Create an agreement Before evaluation (protects all; frees user) Ulab Team WHO – USERS & ULAB TEAM 23

24. WHAT – USABILITY EVALUATION 1. Conduct a planning meeting involving the ULAB team. Definition of the usability goals and concerns for the evaluation Establishment of the parts of the evaluation Creation of a user profile Development of screening questionnaire Creation of task scenarios (task scripts) Determination of the quantitative and qualitative measures (evaluation metrics) Assignment of team roles for the evaluation Establishment of the method of analysis of data and the baseline criteria Establishment of equipment list needed for the evaluation and configuration of the evaluator room 2. Complete independent tasks to be performed by team members including participant selection criteria, development of test materials, etc. 24

25. WHAT – USABILITY EVALUATION (CONTINUED) 3. Conduct a lab walk-through for the planned evaluation with all team members. Do a rehearsal with one of the team members as the user. This data is not valid.. Edit all materials accordingly 4. Conduct a pilot evaluation. Edit all materials accordingly. 5. Conduct the evaluation with 6 true users. Analyze the results, prepare findings and recommendations. Prepare a final project summary report and deliver the report, all test materials, and raw capture data. Prepare a PowerPoint presentation of key findings. 25

26. Establishment of the project time line Decide the Duration of the Evaluation Sessions Create an Evaluation Timetable – sessions, evaluation, reporting Decide the metrics to capture & the baseline criteria Look at Table 6.1 Possible Measurement Criteria Preparing Task Descriptions – the tasks the participant will perform while interacting with the prototype during the evaluation Task Cards Example - Task Descriptions for Global Warming WHEN - CREATING A TIMETABLE 26

27. Where Will You Do the Evaluation? Field Studies – user’s own environment Controlled Studies – other than user’s environment The Setting for the Global Warming Evaluation Arranging Usability Evaluation Sessions The Arrangements for the Global Warming Evaluation WHERE 27