"Think-Aloud" Usability Test NetSmartz.org Inquiry Topic: Child Online Safety & Parental Measures Presented by: Dixie Davis Laurie Miller Mark Pella Francis Tan EME 6635: Inquiry & Measurement Dr. Vanessa Dennen Spring 2008

Increasing growth of children (ages 5-17) using the Internet for school and entertainment. Problem: Greater access to the Internet has exposed children to objectionable & dangerous content (pornography, predators, violence, identity theft). Question: What are parents doing to keep their children safe when they are using the Internet at home? How effective are these measures? Inquiry Background

Usability Testing Focus 70% of parents confident in ability to gather Internet safety information Assess parental ability to locate materials concerning online safety for children using the Internet. Resource Website Focus:

Protocol Script Welcome: "Thank you for participating in today’s usability test. This is an exciting study that we are conducting to research child safety online. This usability test helps assess parental ability in locating online resources that contain online safety materials. It should not take long to complete. We are testing the site, not you. We are conducting this as a “think aloud” protocol. You will speak your thoughts aloud while completing the task. I will observe and take notes." Agenda: "There are a few steps to today’s usability test. First, I would like to gather your demographic background (used survey questions) Second, you will be asked to complete three separate, but related, tasks using a web browser. You will discuss aloud your thought process as you work in completing the task objective." Entrance Questions: "Do you have any questions that need clarification from me? I want to ensure your comfort and understanding of your role in the study. I want to make this a rewarding experience for you, so do not hesitate to ask if you need anything." Exit Questions: "Do you have any questions for me? Any general comments regarding the site? Regarding this test? Thank you for your time & willingness to participate. You have made a valuable contribution to our efforts."

Protocol Tasks Task 1: Using a search engine

Protocol Tasks Task 2: Locate & Register for NetSmartz Bulletin

Protocol Tasks Task 3: Locate & Download Safety Pledge

Test Findings “Are you kidding me? HaHa. Where do I even begin?” “This looks like a crayon box threw-up. ” – Test Subject A

Test Subjects Three females, one male. Ages ranged from 34 (youngest) to 41 (oldest). Three with post graduate degrees, one a high school graduate. Two with IT backgrounds.

Success Rate: 100% All four subjects able to complete the three tasks. On average, testers spent nearly 6 minutes completing tasks. They spent the most time on Task 2: Finding Bulletin. All had initial difficulty finding either the bulletin or the safety pledges.

All initially had difficulty navigating the site. Quick Links – List of Popular Destinations only available from main page. Left-hand menu bar not intuitive. Ah found it! (Sigh) It is so well hidden!” – Test Subject B

Conclusions Test showed parents could use site to sign up for bulletin and download safety pledges. Website could include search box to allow items to be more easily found. “I don’t know if you’re looking for suggestions, but I wish they had a key word box.” – Subject C. Quick Links list could be included on all topic pages, not just main page.

Reflections Overall the Usability test was more fruitful than many of us thought. Maybe we should have done a hybrid of Retrospective and Concurrent Think Aloud Protocols instead of solely Concurrent Think Aloud Protocol. The preparedness level for the administrator of a Usability Test is similar to that of an Interview.

Lessons Learned Reiterate the fact the participants should use the NetSmartz website in the directions for each tasks. Be more specific about what pledge we want them to download It is difficult to not give a hint when the participant is frustrated Following Think -Aloud protocol is difficult for some participants.