1. © 2006 AT&T Knowledge Ventures. All rights reserved. AT&T is a registered trademark of AT&T Knowledge Ventures. Wireless B2B eCommerce Manager (Nano) Usability Test Plan Presented by Bo Lora May 29, 2007

2. © 2006 AT&T Knowledge Ventures. All rights reserved. AT&T is a registered trademark of AT&T Knowledge Ventures. Page 2 Introduction This document describes the usability evaluation plan for the Nano project. The purpose of a usability evaluation is to predict the expected performance of the actual customer using the current product and materials, as well as detect any serious problems prior to the release of the product. This plan includes the following sections: Purpose Of The Usability Evaluation Target Audience Design Of The Usability Evaluation Data Collection Methodology Deliverables Resources Schedule

3. © 2006 AT&T Knowledge Ventures. All rights reserved. AT&T is a registered trademark of AT&T Knowledge Ventures. Page 3 Introduction What is Nano? Nano is a proposed solution for managing one aspect of AT&T B2B Wireless success: the need for improved website building and data management within the ATG / Premier data schema. Due to the increased demand for Wireless B2B websites, the Premier Profile team has recently experienced more website requests than they can create within their SLA (Service Level Agreement). Design Usability Goals The evaluation will focus on determining if the needs of the user are met in a easy to understand, useful, and productive manner. Specific measurable goals for the usability evaluation are outlined in the Usability Evaluation Goals section of this document.

4. © 2006 AT&T Knowledge Ventures. All rights reserved. AT&T is a registered trademark of AT&T Knowledge Ventures. Purpose of the Usability Evaluation The usability evaluation of the Wireless B2B eCommerce Manager application will evaluate the potential for errors and difficulties involved in using the application for human resource activities. The areas that will be tested through the usability evaluation process were identified by the project team and profile team management.

5. © 2006 AT&T Knowledge Ventures. All rights reserved. AT&T is a registered trademark of AT&T Knowledge Ventures. Page 5 Concerns This section outlines the specific concerns that the Profile team, development team, and Web Solutions team may have. Can users successfully navigate through the application? Is the information logically organized and grouped for the novice Profile team member? Can the application be used with only the on-line help, or is a paper-based user guide required? How will users feel about using the on-line help? Is context- sensitive help a requirement for our users? Are there tasks that users will want to perform that are not currently supported by the Wireless B2B eCommerce Manager Application?

6. © 2006 AT&T Knowledge Ventures. All rights reserved. AT&T is a registered trademark of AT&T Knowledge Ventures. Page 6 Usability Evaluation Goals Specific usability goals were determined from the previous concerns. These goals allow for the creation of evaluation scenarios and tasks that will let us know if our concerns are valid and what measures can help us determine if in fact the participants are having trouble completing the tasks. This evaluation will be based on the participants ability to: Begin using the application with no documentation. Locate a specific B2B site in less than a minute. Create a B2B site in 5 minutes or less, with no assistance from a Senior Profile Team member. Move from one section to another with no expressed, or visible difficulty. Find related information with no expressed or visible frustration. Have no more than two false attempts in finding specific information.

7. © 2006 AT&T Knowledge Ventures. All rights reserved. AT&T is a registered trademark of AT&T Knowledge Ventures. Page 7 Usability Evaluation Goals We will also use a survey to determine subjective reactions: Users feel that the pictures used on the icons are recognizable and do facilitate system use/understanding? Users feel comfortable using only the on-line help, or if context-sensitive help is a requirement. Users feel the on-line help provides them with all of the information necessary to use the system. Users feel that the on-line reference answers all of their B2B site creation/maintenance questions.

8. © 2006 AT&T Knowledge Ventures. All rights reserved. AT&T is a registered trademark of AT&T Knowledge Ventures. Page 8 Proposed Scenarios The following specific scenarios will be used to evaluate the proposed application: Find a site by FAN Find a site by Keyword Create a B2B Site Add user to B2B Site Restrict Motorola phone in buy flow of a B2B Site Add Promotion to B2B site Change default shipping Disable B2B site Create a Blackberry only version of a B2B Site Change welcome message Change B2B site contact information Change B2B site to browse only Add a new rate plan

9. © 2006 AT&T Knowledge Ventures. All rights reserved. AT&T is a registered trademark of AT&T Knowledge Ventures. Target Audience The selection of participants whose background and abilities are representative of the products intended end user is a crucial element of the evaluation process. For this reason, valid results will be obtained only from active profile team members who have daily tasks of creating and maintaining B2B sites.

10. © 2006 AT&T Knowledge Ventures. All rights reserved. AT&T is a registered trademark of AT&T Knowledge Ventures. Page 10 Participant Selection Criteria The following list shows the key characteristics of the end users that are considered as critical differentiators for successful adoption, and use, of the Wireless B2B eCommerce Manager. These characteristics are the basis for participant selection for the usability evaluation. The participants will be selected to reflect the range of characteristics shown below. Senior Profile Analyst: Someone who has an extensive knowledge of ACC and is involved in troubleshooting high priority outages. Profile Analyst: Someone who is involved in the daily task of creating and maintaining Premier profiles. Junior Profile Analyst: Someone who is fairly new to the Profile team.

11. © 2006 AT&T Knowledge Ventures. All rights reserved. AT&T is a registered trademark of AT&T Knowledge Ventures. Design of the Usability Evaluation A single usability evaluation will be run in at least 10 individual participant sessions. Each individual session will consist of a set of tasks and an interview/questionnaire for the participants to complete.

12. © 2006 AT&T Knowledge Ventures. All rights reserved. AT&T is a registered trademark of AT&T Knowledge Ventures. Page 12 The Evaluation Process Participant greeting and Orientation Each participant will be personally greeted by the evaluator and made to feel comfortable and relaxed. The participants will receive a short, verbal scripted introduction and orientation to the evaluation. This introduction will explain the purpose and objective of the evaluation, the need for product anonymity until after the evaluation, and additional information about what is expected of them. They will be assured that the product is the center of the evaluation and not themselves, and that they should perform in whatever manner is typical and comfortable for them. The participants will be informed that they are being observed and videotaped.

13. © 2006 AT&T Knowledge Ventures. All rights reserved. AT&T is a registered trademark of AT&T Knowledge Ventures. Page 13 The Evaluation Process Performance evaluation The performance evaluation consists of a series of tasks that are evaluated separately and sequentially. The individual participants complete the tasks while being recorded on video and observed by the usability specialists. The scenario is as follows: – After the orientation, the participants will be asked to sit down at the computer. – The evaluator will give the participants defined tasks and instruct them go forward through each task verbalizing their thoughts out loud. – The participants will be encouraged to work without guidance and in case they become stuck or hopelessly confused the evaluator will note these occurrences and may ask questions to probe and help to pinpoint the cause of the problem.

14. © 2006 AT&T Knowledge Ventures. All rights reserved. AT&T is a registered trademark of AT&T Knowledge Ventures. Page 14 The Evaluation Process Participant debriefing After all tasks are complete or the time expires, each participant will be debriefed by the evaluator. The debriefing will be taped and will include the following: – Completion of a brief post evaluation questionnaire in which the participants share their opinions on the products usability, appearance of application screens, and general impressions of the application. – Participants overall comments about his or her experience – Participants responses to probes from the evaluator about specific errors or problems encountered during the evaluation The debriefing session serves several functions: – It allows the participants to say whatever they like, which is important if tasks are frustrating. – It provides important information about each participants rationale for performing specific actions. – It allows the collection of subjective preference data about the application and its supporting documentation.

15. © 2006 AT&T Knowledge Ventures. All rights reserved. AT&T is a registered trademark of AT&T Knowledge Ventures. Page 15 Logistics Location: We will simulate a typical office environment during the usability evaluation. We will utilize an office (Bothell 6 – 3023A) that will be large enough to comfortably accommodate a desk for the participant to sit at while completing the evaluation. The prototype application will be run on a docked laptop with external keyboard and mouse. Prototype: The prototype employed for this evaluation will be a stand alone html prototype that requires no network connection. This prototype will be representative of the look and feel of the final product and will adhere to current AT&T Intranet Guidelines. Observation: Each usability evaluation will be set up in eMeeting. A meeting room with a projector and speakers. The session will be recorded via screen capture on the laptop with an external microphone. A video camera will be set up to record the participants facial actions and comments. If possible, a video signal will be fed to the meeting room to provide observation of participant.

16. © 2006 AT&T Knowledge Ventures. All rights reserved. AT&T is a registered trademark of AT&T Knowledge Ventures. Page 16 Data Collection Methodology Data will be collected through the use of a thinking aloud protocol. Measures to be collected include the following: The average amount of time to complete each task The percentage of participants who finished each task successfully The number of cases in which the participants were not able to complete a task due to an error from which they could not recover The number of times the participant used the help line or on-line documentation for each task The number of positive or critical statements about the on-line help documentation Number of and types of findings, including: Good. Events when the participant found the application to be intuitive and correct in its approach. Bad. Events when the participant makes a mistake but is able to recover during the task in the allotted time. Catastrophic. Events when the participant makes a mistake and is unable to recover and complete the task on time. The participant may or may not realize a mistake has been made. The number of indications of frustration or joy from the participant The number of subjective opinions of the usability and aesthetics of the product expressed by the participants

17. © 2006 AT&T Knowledge Ventures. All rights reserved. AT&T is a registered trademark of AT&T Knowledge Ventures. Deliverables At the completion of the usability evaluation, a formal analysis will be performed. A final evaluation report and a highlight tape will be provided, which will detail the significant problems and observations detected during the usability evaluation, and recommendations to address the findings, will be delivered to the project team.

18. © 2006 AT&T Knowledge Ventures. All rights reserved. AT&T is a registered trademark of AT&T Knowledge Ventures. Page 18 Schedule The usability evaluation will be conducted in June, 2007. This date is dependent on approval of this evaluation plan by the Profile Team and identification and selection of qualified participants. TimeTue June 12thWed June 13thThus June 14th 9:00 a.m.Pilot SessionShannon HarringtonTerry Reid 1:00 p.m.Victor DiazEugene YonDavid Thai 3:00 p.m.Molly UtsickWilliam RynearsonGladys Woo