2. 1 Evaluation / Usability

3. Usability Is it a “good” interface? In what ways? Usability: How well users can use the system’s functionality How well can the interface be used – it’s just that simple

6. 5 Original

7. 6 Proximity

8. 7 Repetition

9. Examples of Bad Design … and Why Elevator controls and labels on the bottom row all look the same, so it is easy to push a label by mistake instead of a control button People do not make same mistake for the labels and buttons on the top row. Why not?

10. Logical or ambiguous design? Where do you plug the mouse? Where do you plug the keyboard? top or bottom connector? Do the color coded icons help? From: www.baddesigns.com

11. Internal and external consistency Internal consistency refers to designing operations to behave the same within an application Difficult to achieve with complex interfaces External consistency refers to designing operations, interfaces, etc., to be the same across applications and devices Very rarely the case, based on different designer’s preference Most successful in product families (e.g MS Office) Op. Sys. vendors may define style guidelines

12. External Inconsistency … Keypad numbers layout 12 3 456 7 89 7 89 12 3 456 0 0 (a) phones, remote controls(b) calculators, computer keypads

13. Usability Problem Example: Unexpected Occurrence of Events

14. Usability Measures – 5 Often Used Time to learn How long does it take for typical members of the community to learn relevant task? Speed of performance How long does it take to perform relevant benchmarks? Rate of errors by users How many & what kinds of errors are made during benchmark tasks? Retention over time Frequency of use and ease of learning help make for better user retention Subjective satisfaction Do they like it? Allow for user feedback via interviews, free-form comments and satisfaction scales

17. 16 Original

18. compliant interaction state evident in mechanical buttons rotary knobs reveal internal state and can be controlled by both user and machine

19. Evaluation Techniques Evaluation tests usability and functionality of system occurs in laboratory, field and/or in collaboration with users evaluates both design and implementation

20. 19 Questionnaire Design Collect general demographic information that may be relevant Age, sex, computer experience, etc. Evaluate feelings towards your product and other products Important to design questionnaire carefully Users may find questions confusing May not answer the question you think you are asking May not measure what you are interested in

21. 20 Questionnaire, 2 “Likert scale” Propose something and let people agree or disagree: agreedisagree The system was easy to use:1.. 2.. 3.. 4.. 5 “Semantic differential scale” Two opposite feelings: difficult easy Finding the right information was: -2.. -1.. 0.. 1.. 2 If multiple choices, rank order them: Rank the choices in order of preference (with 1 being most preferred and 4 being least): Interface #1 Interface #2 Interface #3 Interface #4 (in a real survey, describe the interfaces)

22. 21 Survey example

23. 22 Where Evaluate? Usability Labs Cameras, 2-way mirrors, specialists Separate observation and control room Should disclose who is watching Having one may increase usability evaluations in an organization Can usually perform an evaluation anywhere Can use portable videotape recorder, etc.

24. Cognitive Walkthrough Proposed by Polson et al. evaluates design on how well it supports user in learning task usually performed by expert in cognitive psychology expert ‘walks though’ design to identify potential problems using psychological principles

25. Cognitive Walkthrough Example: Code walkthrough in Software Engineering To review a segment of program code by the expert other than programmer Sequence of actions Selecting a set of program codes Check certain characteristics, such as, coding style, spelling variables conventions, function declarations, system-wide invariants etc.

26. Cognitive Walkthrough (ctd) For each task walkthrough considers what impact will interaction have on user? what cognitive processes are required? what learning problems may occur? Analysis focuses on goals and knowledge: does the design lead the user to generate the correct goals?

27. 26 Questions Cognitive Walkthrough 1. Is the next goal clear at this stage? 2. Is the appropriate action obvious? 3. Is it clear that this action leads to the goal? 4. What problems are there in performing the action?

28. Cognitive Walkthrough: How (cont.) Walk through the task while answering these Questions: Will the user know what to do? Will the user see how to do it? Will the user understand from feedback whether their action was correct?

29. Heuristic Evaluation Proposed by Nielsen and Molich. usability criteria (heuristics) are identified design examined by experts to see if these are violated Example heuristics system behaviour is predictable system behaviour is consistent feedback is provided Heuristic evaluation `debugs' design.

30. The Procedure Several independent evaluators each uses the same checklist each works alone each makes a list of usability problems Combine lists into a single list works well as a group activity

31. HE: 10 Usability Heuristics 1. Visibility of system status The system should always keep users informed about what is going on, through appropriate feedback within reasonable time 2. Match between system and the real world The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order 3. User control and freedom Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo

32. HE: 10 Usability Heuristics 4.Consistency and standards Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions 5.Error prevention Even better than good error messages is a careful design which prevents a problem from occurring in the first place 6.Recognition rather than recall Make objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate

33. HE: 10 Usability Heuristics 7.Flexibility and efficiency of use Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions 8.Aesthetic and minimalist design Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility 9.Help users recognize, diagnose, and recover from errors Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution

34. HE: 10 Usability Heuristics 10. Help and documentation Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large Jacob Nielsen originally developed the heuristics for heuristic evaluation in collaboration with Rolf Molich in 1990. Nielsen since refined the heuristics based on a factor analysis of 249 usability problems to derive a set of heuristics with maximum explanatory power, resulting in this revised set of heuristics

35. 34 “Think Aloud” Protocols “Single most valuable usability engineering method” Find out why user does things What thought would happen, why stuck, frustrated, etc. Encourage users to expand on whatever interesting But interferes with timings May need to “coach” user to keep talking Unnatural to describe what thinking Ask general questions: “What did you expect”, “What are you thinking now” Not: “What do you think that button is for”, “Why didn’t you click here” Will “give away” the answer or bias the user Alternative: have two users and encourage discussion

36. Physiological measurements emotional response linked to physical changes these may help determine a user’s reaction to an interface measurements include: heart activity, including blood pressure, volume and pulse. electrical activity in brain: electroencephalogram (EEG)

37. Eye tracking head or desk mounted equipment tracks the position of the eye eye movement reflects the amount of cognitive processing a display requires measurements include fixations: eye maintains stable position. Number and duration indicate level of difficulty with display saccades: rapid eye movement from one point of interest to another scan paths: moving straight to a target with a short fixation at the target is optimal

39. Reality …

40. 39 Example: prompts What is a DTIC user code and how to get one?

41. 40 Example: prompts (Print)

42. Keystroke-Level Model (KLM)

43. KLM Operators KPress a key or button PPoint to a target on the display HHome hands on input device DDraw a line segment MMentally prepare for an action R(system response time)

44. Example: Replacing a Word

45. Operator Estimates Keystroke determined by typing speed 0.28s for average typist (40 wpm), 0.08s for fast typist (155 wpm), 1.20s for worst typist Pointing determined by Fitts’ Law (or general approximation) T = a + b log (d/s +1) OR T = 1.1s Drawing determined by Steering Law T = a + b (d/s)

46. Operator Estimates Homing estimated by measurement T = 0.36s (between keyboard and mouse) Mental prep estimated by measurement T = 1.35s (estimated by taking the total task time, subtracting physical operator time, and dividing by the number of “chunks” of activity) Adapted from Rob Miller

47. Example: Deleting a Word Using Shift-Click M P [start of word] K [click] M P [end of word] K [shift] K [click] H [to keyboard] M K [Del] Total: 3M + 2P + 4K = 7.37 sec Using Delete M P [start of word] K [click] H M K [Del] x n [length of word] Total: 2M + P + H + (n+1) K = 4.44 + 0.28n sec

48. In-class Exercise Generate a KLM model for deleting a file from your desktop Estimate the time it would take using the provided operator times Compare the predicted time with the actual time

49. Example Consider the searching a Word document for all occurrences of a four-letter word, and replacing it with another four-letter word. 48

50. Reach for mouseH[mouse]0.40 Move pointer to "Replace" button P[menu item]1.10 Click on "Replace" command K[mouse]0.20 Home on keyboardH[keyboard]0.40 Specify word to be replaced M4K[word]2.15 Reach for mouseH[mouse]0.40 Point to correct fieldP[field]1.10 Click on fieldK[mouse]0.20 Home on keyboardH[keyboard]0.40 Type new wordM4K[word]2.15 Reach for mouseH[mouse]0.40 Move pointer on Replace-all P[replace-all]1.10 Click on fieldK[mouse]0.20 Total10.2 49

51. 50 Error Messages Blame the system, not the user “Unrecognized” vs. “illegal” command Easy error recovery Can have multiple levels of messages E.g. in XXX product, “can't save file” — why not?

52. Problem: Geek-speak: Refers to “Print dialog”, a technical UI component term Microsoft Word 98 Print Status Dialog Box

53. Problem: Two buttons (stop/cancel) to end the print?? Problem: Two buttons (stop/cancel) to end the print??

54. Eudora Pro 4.0 Special menu Problem: What’s the difference between Find and Search? Problem: What’s the difference between Find and Search?

55. Virus Scan 3.0.3 Warning Problem: Careless writing. Note typo -- “more that” Problem: Careless writing. Note typo -- “more that”

56. Microsoft Notepad Menu Problem: No keyboard equivalents Problem: No keyboard equivalents

57. Microsoft Windows Ports Control Panel Problem: Dialog box control buttons mixed with setting and help buttons

58. CS5540 Eudora Version Installation Dialog Box Problem: Traps user: no way to Cancel installation at this point. Gives options for bad modes.

59. CS554058 SpellTools User Dictionary … Problem: No “...” on Import button

60. CS554059 Oracle’s ORS CLEATS web-application Problem: Should write out “Pin Number,” not “Pin No”

61. Unsubscribe from mailing list Problem: Silly message

62. 61 Bad Error Messages

63. 62 More bad error messages!

64. 63 Another Bad Example http://stinet.dtic.mil/

65. 64 Another Bad Example

66. 65 Another Bad Example

67. 66 Pretty Good Example Pretty Good: travel.yahoo.com: Says what to do to fix it But language is inconsistent

68. 67 Help and Documentation True walk up and use? Most people will not read documentation If do, then First time product is used, or else In a panic, so need information right away Iterative design of documentation needed SuperBook application answer found in 4.3 minutes, compared to 7.6 minutes before fixing Help system is an extra feature to learn “Help doesn’t” If need to add help, maybe fix the feature? Use documentation writers to help refine the system Good quality writing

69. 68 Good Help Example NSF report system What & Why

70. Firefox: No Zoom Indicator 69

71. Worst interfaces 70

72. 71

73. Kitchen timer 72

74. Turn down the TV 73

75. Xerox Real Business (worst 2011) 74

76. Examples of non-usable systems Eclipse Hard learning curve, bad for beginners Too many windows, confusing layout, too much going on (advanced options)

77. ISIS pops up a window, pop-up blocker problems can’t use browser’s back button gives course instructor number instead of name must enter schedule-number instead of clicking on course ID and number Must use semester as a code, and if you get this wrong it’s confusing what’s wrong Interface has pictures/images for links, not the conventional under-lined links not integrated into COD, so must copy info over from COD to ISIS Kicks you off too quickly Too often overloaded, too many users Unavailability at certain hours / don’t know when it’s unavailable / status not visible easily Links on main page are too similar SSN number use

78. 77 175 design Guidelines These are primarily intended to guide you to a good design, not to be used for evaluation 1)Help users plan goals, tasks 2)Provide a clear model of how users view system in terms of tasks 3)Help users with system model, metaphors, work context 4)Design to match user’s conception of high-level task organization 5)Help users understand what system features exist and how they can be used in their work context 6)Help users decompose tasks logically 7)Make clear all possibilities for what users can do at every point 8)Keep users aware of system state for planning next task 9)Keep the task context visible to minimize memory load 10)Help users plan the most efficient ways to complete their tasks 11)Keep users aware of task progress, what’s been done and what’s left to do 12)Provide cognitive affordances at the end of critical tasks to remind users to complete the transaction 13)Provide effective cognitive affordances that help users get access to system functionality 14)Help users know/learn what actions are needed to carry out intentions Quoted from Chapter 19 of H. Rex Hartson and Pardha S. Pyla, The UX Book: Ensuring a Quality User Experience, to be published by Morgan Kaufmann / Elsevier in 2011.

79. Guidelines, cont. 15)Help users know how to do something at action/object level 16)Help users predict outcome of actions 17)Help users determine what to do to get started 18)Provide a cognitive affordance for a step the user might forget 19)Support user with effective sensory affordances in presentation of cognitive affordances 20)Make cognitive affordances visible 21)Make cognitive affordances noticeable 22)Make text legible, readable 23)Control cognitive affordance presentation complexity with effective layout, organization, and grouping 24)Present cognitive affordance in time for it to help the user before the associated action 25)Help user determine actions with effective content/meaning in cognitive affordances 26)Design cognitive affordances for clarity 27)Use precise wording in labels, menu titles, menu choices, icons, data fields 28)Use a verb and noun and even an adjective in labels where appropriate. 29)Avoid vague, ambiguous terms. 78 Quoted from Chapter 19 of H. Rex Hartson and Pardha S. Pyla, The UX Book: Ensuring a Quality User Experience, to be published by Morgan Kaufmann / Elsevier in 2011.

80. Guidelines, cont. 30)Be as specific to the interaction situation as possible 31)Clearly represent work domain concepts 32)Use dynamically changing labels when toggling 33)Provide cognitive affordances to indicate formatting within data fields 34)Constrain the formats of data values to avoid data entry errors 35)Provide clearly marked exits 36)Provide clear “do it” mechanism 37)Be predictable; help users predict outcome of actions with feed-forward information in cognitive affordances 38)Make choices distinguishable 39)Be consistent with cognitive affordances 40)Use consistent wording in labels for menus, buttons, icons, fields 41)Use similar names for similar kinds of things 42)Do not use multiple synonyms for the same thing 43)Use the same term in a reference to an object as the name or label of the object 44)Use different terms for different things, especially when the difference is subtle 79 Quoted from Chapter 19 of H. Rex Hartson and Pardha S. Pyla, The UX Book: Ensuring a Quality User Experience, to be published by Morgan Kaufmann / Elsevier in 2011.

81. Guidelines, cont. 80 45)Be consistent in the way that similar choices or parameter settings are made 46)Decompose complex instructions into simpler parts 47)Use appropriate layout and grouping by function to convey content and meaning 48)Group together objects and design elements associated with related tasks and functions 49)Do not group together objects and design elements that are not associated with related tasks and functions 50)Support user choices with likely and useful defaults 51)Provide the most likely or most useful default selections 52)Offer most useful default cursor position 53)Relieve human short term memory loads by maintaining task context visibly or audibly for the user 54)Support human memory limits with recognition over recall 55)Avoid requirement to retype or copy from one place to another 56)Support special human memory needs in audio interaction design 57)Avoid cognitive indirectness 58)Be complete in your design of cognitive affordances; include enough information for users to determine correct action Quoted from Chapter 19 of H. Rex Hartson and Pardha S. Pyla, The UX Book: Ensuring a Quality User Experience, to be published by Morgan Kaufmann / Elsevier in 2011.

82. Guidelines, cont. 81 59)Prevent loss of productivity due to hesitation, pondering 60)Use enough words for unambiguous labels 61)Add supplementary information, if necessary 62)Give enough information for users to make confident decisions 63)Give enough alternatives for user needs 64)Employ usage-centered wording, the language of the user and the work context, in cognitive affordances 65)Find ways to anticipate and avoid user errors in your design 66)Help users avoid inappropriate and erroneous choices 67)Disable buttons, menu choices to make inappropriate choices unavailable 68)Gray out to make inappropriate choices appear unavailable 69)But help users understand why a choice is unavailable 70)Provide a clear way to undo and reverse actions 71)Offer constructive help for error recovery 72)Avoid confusing modalities 73)Distinguish modes clearly 74)Use “good modes” where they help natural interaction without confusion 75)Support human memory limitations in the design of task structure 76)Support user with effective task structure and interaction control 77)Provide alternative ways to perform tasks Quoted from Chapter 19 of H. Rex Hartson and Pardha S. Pyla, The UX Book: Ensuring a Quality User Experience, to be published by Morgan Kaufmann / Elsevier in 2011.

83. Guidelines, cont. 82 78)Provide shortcuts 79)Provide logical grouping in layout of objects 80)Group together objects and functions related by task or user work activity 81)But avoid grouping of objects and functions if they need to be dealt with separately 82)Support task thread continuity by anticipating the most likely next task, step, or action 83)Make the most of user’s work 84)Do not requiring users to re-enter data 85)Retain user state information 86)Avoid the feeling of loss of control 87)Give direct manipulation support 88)Always provide a way for the user to “bail out” of an on-going operation 89)Support users making physical actions with effective sensory affordances for sensing physical affordances 90)Support user with effective physical affordances for manipulating objects, help in doing actions 91)Avoid physical awkwardness 92)Accommodate physical disabilities 93)Design layout to support manual dexterity and Fitts’ law Quoted from Chapter 19 of H. Rex Hartson and Pardha S. Pyla, The UX Book: Ensuring a Quality User Experience, to be published by Morgan Kaufmann / Elsevier in 2011.

84. Guidelines, cont. 83 94)Support targeted cursor movement by making selectable objects large enough 95)Group clickable objects related by task flow close together 96)But not too close, and do not include unrelated objects in the grouping 97)Design physical movement to avoid physical overshoot 98)Include physicality in your design when the alternatives are not as satisfying to the user 99)Check your functionality for missing features 100)Check your functionality for non-user-interface software bugs 101)Avoid too much automation and loss of user control 102)Help the user by automating where there is an obvious need 103)Provide feedback for all user actions 104)Provide progress feedback on long operations 105)Request confirmation as a kind of intervening feedback 106)But don’t overuse and annoy 107)Support user with effective sensory affordances in presentation of feedback 108)Make feedback visible 109)Make feedback noticeable 110)Locate feedback within the user’s focus of attention Quoted from Chapter 19 of H. Rex Hartson and Pardha S. Pyla, The UX Book: Ensuring a Quality User Experience, to be published by Morgan Kaufmann / Elsevier in 2011.

85. Guidelines, cont. 84 111)Make feedback large enough to notice 112)Make text legible, readable 113)Control feedback presentation complexity with effective layout, organization, and grouping 114)Help users detect error situations early 115)Maintain a consistent appearance across similar kinds of feedback 116)Maintain a consistent location of feedback presentation on the screen to help users notice it quickly. 117)Use the most effective feedback presentation medium 118)Consider audio as alternative channel 119)Help users understand outcomes with effective content/meaning in feedback 120)Design feedback for clarity 121)Support clear understanding of outcome (system state change), so users can assess effect of actions 122)Give clear indication of error conditions 123)Be complete in your design of feedback; include enough information for users to fully understand outcomes and be either confident that their command worked or certain about why it didn’t 124)Prevent loss of productivity due to hesitation, pondering Quoted from Chapter 19 of H. Rex Hartson and Pardha S. Pyla, The UX Book: Ensuring a Quality User Experience, to be published by Morgan Kaufmann / Elsevier in 2011.

86. Guidelines, cont. 85 125)Add supplementary information, if necessary 126)Give enough information for users to make confident decisions about the status of their course of interaction 127)Help users understand what the real error is 128)Give enough information about the possibilities or alternatives so user can make an informed response to a confirmation request 129)Design feedback wording, especially error messages, for positive psychological impact 130)Make the system take blame for errors 131)Be positive, to encourage 132)Provide helpful, informative error messages, not “cute” unhelpful messages 133)Employ usage-centered wording, the language of the user and the work context, in displays, messages, and other feedback 134)Be consistent with feedback 135)Label outcome or destination screen or object consistently with starting point and action 136)Organize feedback for ease of understanding 137)Provide user control over amount and detail of feedback 138)Give only most important information at first; more on demand Quoted from Chapter 19 of H. Rex Hartson and Pardha S. Pyla, The UX Book: Ensuring a Quality User Experience, to be published by Morgan Kaufmann / Elsevier in 2011.

87. Guidelines, cont. 86 139)Organize information displays for ease of understanding 140)Eliminate unnecessary words 141)Group related information 142)Control density of displays; use white space to set off 143)Columns are easier to read than wide rows 144)Use abstraction per Shneiderman’s “mantra”: Overview first; zoom and filter; details on demand 145)Employ usage-centered wording, the language of the user and the work context 146)Avoid the use of anthropomorphism in interaction designs 147)Avoid using first-person speech in dialogue 148)Avoid condescending offers to help 149)Avoid poor attempts at humor 150)Avoid violent, negative, demeaning terms 151)Avoid use of psychologically threatening terms, such as “illegal”, “invalid”, “abort” 152)Avoid use of the term “hit” instead of “press” or “click” 153)Avoid irritation with annoying sound and color in displays 154)Use color conservatively 155)Use pastels, not bright colors Quoted from Chapter 19 of H. Rex Hartson and Pardha S. Pyla, The UX Book: Ensuring a Quality User Experience, to be published by Morgan Kaufmann / Elsevier in 2011.

88. Guidelines, cont. 87 156)Be aware of color conventions (e.g., avoid red, except for urgency) 157)Watch out for focusing problem with red and blue 158)Avoid fancy or cute design without a real purpose 159)Make presentation of text legible 160)Make font size large enough for all users 161)Use good contrast with background 162)Use mixed case for extensive text 163)Avoid too many different fonts, sizes 164)Use legible fonts 165)Use color other than blue for text 166)Accommodate sensory disabilities and limitations 167)Allow user settings, preference options to control presentational parameters 168)Accommodate different levels of expertise/experience with preferences 169)Don’t let affordances for new users be performance barriers to experienced users 170)Be helpful with Help 171)Do not try to achieve the appearance of simplicity by just reducing usefulness 172)Organize complex systems to make the most frequent operations simple Quoted from Chapter 19 of H. Rex Hartson and Pardha S. Pyla, The UX Book: Ensuring a Quality User Experience, to be published by Morgan Kaufmann / Elsevier in 2011.

89. Guidelines, cont. 88 173)Use consistent layout/location for objects across screens 174)Maintain custom style guides to support consistency 175)Use structurally similar names and labels for objects and functions that are structurally similar 176) ….and the list isn’t even finished Quoted from Chapter 19 of H. Rex Hartson and Pardha S. Pyla, The UX Book: Ensuring a Quality User Experience, to be published by Morgan Kaufmann / Elsevier in 2011.