1. Usability Methods: Cognitive Walkthrough & Heuristic Evaluation Dr. Dania Bilal IS 588 Spring 2008 Dr. D. Bilal

2. Purposes Measures multiple components of the user interface Addresses relationships between system and its users Bridges the gap between human and machines

3. Purposes Measures the quality of system design in relation to its intended users Involves several methods, each applied at appropriate time of the design and development process

4. Usability Attributes As described by Neilsen Learnability Efficiency Memorability Errors & their severity Subjective satisfaction

5. Learnability System must be easy to learn, especially for novice users Hard to learn systems are usually designed for expert users Learning curve for novice and expert users

6. Efficiency System should be efficient to use so that once the user has learned how to use it, the user can achieve a high level of productivity Efficiency increases with learning

7. Memorability System should be easy to remember, especially by casual users No need to learn how to use system all over again after a period of not using it

8. Errors System should have a low error rate System should provide user with a recovery mechanism Minor errors Major errors

9. Minor Errors Errors that did not greatly slow down user’s interaction with the system User is able to recover from them through system feedback through awareness of error made

10. Major Errors Difficult to recover from them Lead to faulty work if high in frequency May not be discovered by the user Errors can be catastrophic

11. Subjective Satisfaction System should be likeable by users (affective) Satisfaction varies with purpose of system user goals

12. Assumptions The designer’s best guess is not good enough The user is always right The user is not always right Users are not designers Designers are not users More features are not always better Minor interface details matter Online help does not really help Source: Nielsen, J. (1993). Usability Engineering. San Diego: Morgan Kaufman.

13. Cognitive Walkthrough Method Involves experts acting on behalf of actual users Characteristics of typical users are identified & documented Tasks focusing on aspects of design to be evaluated are developed

14. Cognitive Walkthrough Method An observer “experimenter” is present Prepares tasks Takes notes Provides help, etc. Coordinates and overseas final report

15. Cognitive Walkthrough Method Expert walkthrough interface on each task Expert records problems that user may experience Assumptions about what would cause problems and why are noted Benchmarks may be used for each task

16. Sample Questions for Walkthrough Will the user know what to do to complete part of or the whole task successfully? Can the user see the button or icon to use for the next action? Can the user find specific subject category from the hierarchy?

17. Cognitive Walkthrough Each expert documents experience about walkthrough for each task Critical problems documented Problems and what cause them are explained Draft report/notes are compiled and shared with other experts and Experimenter

18. Debriefing Session Experts and experimenter meet & discuss findings Experimenter shares his/her observational notes with experts Findings include success stories & failure stories, as applicable Consolidated report is generated

19. Walkthrough Report Include questions experts for each of the tasks and the consolidated answer Use benchmarks and map out the finding for each task See Assignment 4: Usability for additional information on benchmarks

20. Heuristic Evaluation Evaluators interact with an interface several times and map interface to specific heuristics or guidelines Example: Nielsen’s ten heuristics Each evaluator generates a report Reports are aggregated and final report is generated An observer may be present

21. Stages of Heuristic Evaluation Stage 1: Debriefing session Experts told what to do Written instructions provided to each expert Heuristics provided to each expert as part of written instructions Verbal instructions may be included

22. Stages of Heuristic Evaluation Stage 2: Evaluation sessions Each expert tests system based on heuristics Expert may also use specific tasks Two passes are taken through interface First pass: overview and familiarity Second pass: Focus on specific features & identify usability problems

23. Stages of Heuristic Evaluation Stage 3: Debriefing session Experts meet to discuss outcome and compare findings Experts consolidate findings Experts prioritize usability problems found & suggest solutions

24. Neilsen’s Heuristics Ten heuristics found at http://www.useit.com/papers/heuristic/heuris tic_list.html http://www.useit.com/papers/heuristic/heuris tic_list.html Additional rules, see Text. Some heuristics can be combined under categories and given general description.

25. Usability Heuristics http://www.usabilityfirst.com/methods http://www.useit.com/papers/heuristic/heuristic_eval uation.htmlhttp://www.useit.com/papers/heuristic/heuristic_eval uation.html (how to conduct a heuristic evaluation) http://www.uie.com/articleshttp://www.uie.com/articles (collection of articles) http://www.uie.com/articles/usability_tests_learn/ http://www.uie.com/articles/usability_tests_learn/ Learning about usability test (Jared Spool) http://www.useit.com/papers/heuristic/severityrating.htmlhttp://www.useit.com/papers/heuristic/severityrating.html (Severity rating)