1. Slides for User interface design A software engineering perspective Soren Lauesen 13. More on usability testing August 2006 © 2005, Pearson Education retains the copyright to the slides, but allows restricted copying for teaching purposes only. It is a condition that the source and copyright notice is preserved on all the material.

2. Fig 13.1 Usability test - Misunderstandings Not demo of system Not a test with colleague as user Functional system not necessary Usability lab not necessary Purpose of usability testing during development? 1.Find program bugs? 2.Prove that system is easy to use? 3.Find problems? 4.Correct problems? 5.Experience the users? Purpose of usability testing with final system?

3. Fig 13.2 Planning the test Which users:One + revision. Then 2-4 (lots of data). Domain knowledge:E.g. know reception work. IT knowledge:E.g text processing. Finding test users:Actual users; on the street; marketing agent,... Test site:E.g. our meeting room, customer’s site, public area, usability lab. Facilitator:Main contact with user. ”Computer”:Designer - make sure he doesn’t interfere. Log keeper:Notes down - particularly the problems. Test tasks:E.g. book guest A based on a letter. Receive guest B who has booked. Change room for guest C who is checked in. Presenting tasks:E.g. written, acting, explain. Don’t give hints! Start-up state:E.g. system started, guest B booked, guest C checked in. User instruction:As in real life. E.g. short written user’s guide? Colleague’s intro? Course?

4. Test method:E.g. observe one user at a time, think aloud one at a time, two users discussing. Data collection:E.g. written notes (log), tape, video, computer log, usability lab. Debriefing:Good things about the system? Bad things? Do you think the system could... ? Planning the time:Welcome and intro:10 min Test tasks60 min Debriefing10 min Reporting the problems100 min Total, one user3 hours Time for three users?? (Fig 13.2 Cont.)

5. Fig 13.4 Problems and hit-rates Total: 60 problems

6. (Fig 13.4 Cont.)

7. Fig 13.5A Test tasks - planning Task 2: John Simpson arrives to check in. Task 1: John Simpson wants to book a room from Mon-Wed and another from Mon-Tues. Task 3: John Simpson reports next morning that he wants another room. Why is this a bad sequence? An easy task: Record name and address for John Simpson. Why is this a bad test task?

8. Fig 13.5B Test tasks - hidden help? Version A: John Simpson wants to check in. Find him on the FindGuest screen. Double click to open his Stay screen. Version B: John Simpson wants to check in. He has stay number 710. Version C: John Simpson arrives 23rd October. He says there should be two rooms for him. If asked:He cannot remember his booking number (or stay number). He lives 456 Orange Grove, Victoria Australia (can’t remember zip code) He leaves 26th October.

9. Fig 13.7A Test report and analysis Within 12 hours: List all problems: Where in dialogue, by whom. User’s expectation, misunderstanding, etc. User’s proposals (if any) Cross check with log. Don’t think of solutions Classify the problems - severity. After a good night’s sleep: What to change: Severity vs. development cost Severity classes: 1Missing functionality 2Task failure 3Annoying 4Medium problem (succeeds after long time) 5Minor problem (succeeds after short time) 6Setup error

10. Fig 13.7B Log from usability test

11. Fig 13.7C Test report