1. Usability testing

2. Goals & questions focus on how well users perform tasks with the product. – typical users – doing typical tasks. Comparison of products or prototypes common – Old version vs. new version – Previous system vs. new system Focus is on – time to complete task – number & type of errors Data collected by video and interaction logging is used to – calculate performance times – to identify & explain errors User satisfaction questionnaires & interviews provide data about users’ opinions. Field observations may be used to provide contextual understanding.

3. Project Step 6: Summative Evaluation / Usability Testing Review your Requirements Analysis and your Design claims analysis, and generate from that a usability specification containing benchmark tasks and outcomes expected. Be sure to include a good variety of benchmark tasks for usability testing, including some easy tasks and some difficult tasks. Then, run usability tests on your prototype with at least 3 subjects using the think-aloud protocol. Subjects should be from your targeted subject pool if possible.

4. Benchmarks center on the basic usability study measurements: Success/Failure within a time threshold Time on task # of errors before completion

5. Usability testing vs. research Usability testing Improve products Few participants Results inform design Usually not completely replicable Conditions controlled as much as possible Procedure planned Results reported to developers Experiments for research Discover knowledge Many participants Results validated statistically Must be replicable Strongly controlled conditions Experimental design Scientific reported to scientific community

6. Usability lab with observers watching a user & assistant

7. Portable equipment for use in the field

9. Some type of data  Time to complete a task.  Time to complete a task after a specified time away from the product.  Number and type of errors per task.  Number of errors per unit of time.  Number of navigations to online help or manuals.  Number of users making a particular error.  Number of users completing task successfully.

10. Usability engineering orientation  Aim is improvement with each version.  Current level of performance.  Minimum acceptable level of performance.  Target level of performance.

11. Time tests usability of smart phones http://www.youtube.com/watch?v=5UUvqsTKkjQ&feature=channel http://www.youtube.com/watch?v=5Bumpt7Zh7I&feature=channel

12. How many participants is enough for user testing? The number is a practical issue. Depends on: – schedule for testing; – availability of participants; – cost of running tests. Typically 5-10 participants. Some experts argue that testing should continue until no new insights are gained.