1. 1 Test Materials CSSE 376, Software Quality Assurance Rose-Hulman Institute of Technology April 30, 2007

2. 2 Outline Data Collection Nondisclosure/Consent Form Task Scenarios

3. 3 Data Collection (1/2) Performance data Time: to complete task to reach level of competence to recover from an error Counts: errors successful task completion omitted steps

4. 4 Data Collection (2/2) Preference data Preference for one version rather than another Suggestions for improving product Rankings of product

5. 5 Methods of Data Collection Data loggers Self-reporting Manual collection

6. 6 Coding Schemes CodeEvent BBegin a task EEnd a task PPrompted by test monitor XIncorrect action ?Probe participant during debriefing

7. 7 Nondisclosure/Consent Form Nondisclosure Participant promises not to reveal information about new products Consent Participant agrees to be videotaped Testers agree not to misuse information about participants

8. 8 Cartoon of the Day

9. 9 Task Scenarios Provide realistic scenarios, complete with motivations to perform Sequence tasks in most likely order Match tasks to participant experience Avoid jargon and slang Provide substantial amount of work

10. 10 Presenting the Scenarios Reading scenarios to participants One less thing for participants to do Can control pace Participants read scenarios themselves Less influence of test monitor May need to prevent them from reading ahead (might display tasks on a computer screen)