1 Usability Analysis n Why Analyze n Types of Usability Analysis n Human Subjects Research n Project 3: Heuristic Evaluation

2 Why Perform Usability Analyses n To provide feedback to designer regarding design decisions n To identify and determine solutions to unforeseen problems n To determine the what is required in instruction manuals and/or online help or to test its effectiveness n To determine how well the product meets the design goals

3 Approaches to Usability Testing n Testing Typical users performing typical tasks using the system Results are examined to determine how well the system supports the user n Inspection Experts in usability examine the system interface n Inquiry Evaluators get information from the perspective users to evaluate the system usability Adapted from

4 Testing n Performance Measurement To obtain quantitative data about performance doing tasks Good for testing alternative or against benchmark performance n Retrospective Video data is collected and reviewed with the test user Questions are asked about what was done and why Generally used in conjunction with other methods n Think-Aloud Users asked to share their thoughts about interaction with the system while performing predetermined tasks Helps in understanding the users mental model Adapted from

5 Inspection n Cognitive Walkthrough Group of evaluator evaluate the system by going through a set of tasks Generally used during design phase n Feature Inspection Each feature is analyzed separately to determine its accessibility, learnablity, and other usability characteristics based on various use scenarios Often done near the documentation phase n Heuristic Evaluation Evaluators attempt to determine if the design adheres to general design principles and rules of thumb Generally performed in prototype phase Adapted from

6 Inquiry n Focus Groups Groups of users brought together to discuss issues relating to the system Evaluator plays the role of moderator n Logging Actual Use Computer automatically collect use statistics about the system Details how users perform their work n Questionnaires Used to obtain subjective data from a group of users Adapted from

7 Human Subjects Protections n Began with Nuremberg Code after WWII "the voluntary consent of the human subject is absolutely essential" Informed consent is the cornerstone of human subjects research n In US, regulations went into effect May 30, 1974 Respect for the persons –Recognition of personal dignity and autonomy Beneficence –Maximize benefits and minimize risks Justice –Fair Distribution of benefits and burden Adapted from DHHS Guidebook

8 Human Subjects Research n Institutional Review Boards Group of individuals responsible for reviewing proposed research involving human subjects –To protect human subjects –Ensure Compliance with Federal Law n Informed Consent Document Must lay out purpose of research, time commitment, potential risks and benefits amongst other material to ensure participant has sufficient information to provide informed consent n Training and Certification of Investigators Workshops Online Training –

9 Project 3: Heuristic Evaluation n Heuristic = “a guideline or general principle or rule of thumb that can guide a design decision or be used to critique a decision that has already been made” n Requires multiple Evaluators (3-6) but no users n Each individual evaluator inspects the interface alone Adapted from

10 Project 3: Heuristic Evaluation n To aid in discovering usability problems, use of a list of heuristics is used to generate ideas while evaluating the system n Heuristics used are dependent upon the system and interface being evaluated. n After all evaluators have completed their individual inspections, the evaluators are then allowed to communicate and have their findings aggregated. Adapted from

11 Project 3: Requirements n Step 1: Define General Tasks supported n Step 2: Identify Heuristics that address the tasks n Step 3: Evaluate System n Step 4: Become Certified in Human Subjects Research

12 Define General Tasks supported n Based on current design, identify tasks involved in using interface n These will be the tasks that you will be evaluating n Consider the criteria for judging performance and success

13 Identify Heuristics that address the tasks n Review heuristics provided with assignment, the textbook and the link to Tog’s first principles. n Develop a list of heuristics or principles that you feel are best suited to your tasks n Number of applicable heuristics will vary between projects.

14 Evaluate System n Each person in group should apply the heuristics independently n As a group, identify specific design improvements Clearly identifies a specific problem Offers a solution that reflects a design heuristic Identify each suggestion by the design heuristics that motivated it

15 Become Certified in Human Subjects Research n Go through the training n Provide a certificate for each member of your groups