Specifying Usability Requirements: The Need for Standards Mary Theofanos Brian Stanton Visualization and Usability Group

How do you specify usability requirements?

Despite usability horror stories Usability requirements are rarely identified by organizations purchasing or developing software

What is the state of usability in Government procurements? Federal agencies have virtually no visibility of software product usability before we make procurement decisions We do not know how to include usability requirements in procurements. We do not know how to compare product usability or to plan for or measure usability costs

But Usability can be objectively defined and measured This implies that we can: 1. Identify usability requirements 2. Measure usability before we deploy or purchase a product

ISO Standards define 3 measures of Usability Effectiveness -- a measure of user productivity, how well a user can perform his job accurately and completely. (i.e.: completion rate, number of errors) Efficiency -- a measure of how quickly a user can perform work, the resources expended to accomplish the task. (i.e.: time on task) Satisfaction--The degree to which users like the product – a subjective response in terms of ease of use, frustration, and usefulness. Usability: The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use

Example of an Operational Definition for Usability for a travel system: On their first try, within 15 minutes, 75% of Government travelers shall be able to correctly-  Create a travel request form  Select one departure flight and one return flight  Designate one hotel  Reserve one rental car  Forward the travel request form for approval.. By their second try, within 15 minutes, 90% shall be able to complete the tasks correctly

In 1998, Industry approached NIST to develop a method for factoring usability into procurement decisions Goal: Increase the visibility of software usability  Reduce uncontrolled overhead costs of software usability problems, while improving user productivity and morale.  Encourage software suppliers and consumer organizations to work together to understand user needs and tasks.  Define and validate an industry-wide process for providing visibility of software usability to support product decision-making.

Participants span industry, government and academia Government Brookhaven National Labs Census Bureau DISA DHHS GSA IRS Library of Congress OCLC OPM SSA USDA US Army Corps of Engineers US Air Force State of Georgia, DOT Universities Dalian Maritime Univ. China Rensselaer Polytechnic Inst. University of Indiana San Jose State SEI/CMU Shizuoka University Syracuse University UC Berkeley UCLA University of Maryland University of Michigan University of Copenhagen University College London University of Bologna Industry Boeing Microsoft Oracle Dell Fidelity Investments Motorola Apple General Electric Ford Honeywell Phillips Whirlpool SAP State Farm Xerox

The role of NIST has been to: Act as a facilitator in bringing together industry usability professionals, academics, and government representatives. Maintain the documents and support the ISO standardization process Collect and analyze data to determine the value of incorporating usability into product decision-making and the impact of the adoption of IUSR “products”.

How do we incorporate usability requirements into the procurement process?

The Common Industry Specification for Usability Requirements Defining usability requirements in sufficient detail to make an effective contribution to design and development Defining usability criteria that can be empirically validated subsequently if needed. Developed by NIST and IUSR to provide a structure for:

CISUR supports communication between usability professionals Procurers can specify usability in a Request for Proposals or a contract for software Supplier organizations can determine if usability requirements specified by a customer are realistic for their product, and plan on how to ensure that a product meets these requirements. Suppliers can assess if the usability requirements specified for product development meet the needs of the customer organizations.

CISUR supports communication within and between organizations Among members of the development team to specify requirements for use by the development team Between the customer and supplier of a custom product to define specific customer requirements Between a range of potential customers and a supplier of an off the shelf product, to define diverse requirements.

The CISUR identifies 3 components to specifying requirements Context of Use: description of intended users, their goals, equipment, and environment in which product will be used Performance and satisfaction criteria: ways in which the usability of the product can be measured Test Methods: how the product will be tested to determine whether the usability requirements have been met

The CISUR identifies 3 levels of specification for the components

The CISUR was developed with the following characteristics: Independent of specific design process Facilitates iterative development of requirements Complements other user centered design standards Only applies to usability requirements Does not specify a format for specification As a complement to the CIF

Next Steps 1. ISO Standardization 2. Case Studies

Getting a copy of the CISUR

Questions? Mary Theofanos NIST (301) Brian Stanton NIST (301)