1. Human Computer Interaction Lecture 15 Usability Evaluation

2. Evaluation Techniques
tests usability and functionality of system
occurs in laboratory and/or field (in collaboration with users)
should be considered at all stages during design
Goals of Evaluation
assess extent of system functionality
assess effect of interface on user
identify specific problems

3. Styles of Evaluation Laboratory studies Field Studies
Users are taken out of their normal work environment to take part in controlled tests, often conducted in a specialist usability laboratory
Field Studies
Designers or evaluators are taken out into the user’s work environment in order to observe the system in action

4. Laboratory studies Advantages: Disadvantages: Appropriate
specialist equipment available
uninterrupted environment
Disadvantages:
lack of context
difficult to observe several users cooperating
Appropriate
if system location is dangerous (critical) or impractical and system is constrained single user system to allow controlled manipulation of use

5. Field Studies Advantages: Disadvantages: Appropriate
natural environment
context retained
longitudinal studies possible
Disadvantages:
Distractions
noise
Appropriate
where context is crucial for longitudinal studies

6. Cognitive Walkthrough Heuristic Evaluation
Evaluating Designs
Cognitive Walkthrough
Heuristic Evaluation

7. Cognitive Walkthrough
Proposed by Polson
evaluates design on how well it supports user in learning task i.e. the focus is on the learnability aspect of usability.
usually performed by expert in cognitive psychology
expert ‘walks though’ design to identify potential problems using psychological principles
forms used to guide analysis

8. Cognitive Walkthrough
To do a walkthrough, four things are needed:
A specification or prototype of the system
A Description of the task the user is to perform on the system
A complete, written list of the actions needed to complete the task with the system
An indication of who the users are and what kind of experience and knowledge the evaluators can assume about them

9. Cognitive Walkthrough
The evaluators step through the action sequence (identified in step 3 above) to evaluate the system usability
They answer following questions:
Is the effect of the action the same as the user’s goal at that point?
Will users see that the action is available? (visibility)
Once users have found the correct action, will they know it is the one they need? (clear meaning and effect)
After the action is taken, will users understand the feedback they get? (Understanding)

10. Cognitive Walkthrough (ctd)
Documenting the walkthroughs
To keep track of what is good and what needs improvement
Some standard forms
The cover form
Steps 1-4, authors, date and time
Usability problem report sheet
System, Authors, date, description, severity (harshness)
Analysis focuses on goals and knowledge: does the design lead the user to generate the correct goals?

11. Heuristic Evaluation Proposed by Nielsen and Molich.
Widely used evaluation method
Heuristic evaluation `debugs' design.
First, usability criteria (heuristics) are identified
design judged by a small set of experts (usually between 3 and 5) for compliance with the established usability criteria
Experts evaluate design separately and then meet each other to compile their findings

12. Heuristic Evaluation Example heuristics
Is the status of system visible?
Is the design consistent?
Is feedback provided?

13. Nielsen’s Ten Heuristics
Visibility of system state.
Match between system and the real world.(metaphor)
User control and freedom. (undo/redo)
Consistency and standards.
Error prevention. (Close the doors for errors)
Recognition rather than recall.
Flexibility and efficiency of use. (experienced and inexperienced users)
Aesthetics and minimalist design. (less and relevant info)
Help users recognize, diagnose and recover from errors.
Help and documentation.