Designing Interactive Edits for U.S. Electronic Economic Surveys and Censuses: Issues and Guidelines Elizabeth M. Nichols, Elizabeth D. Murphy, Amy E. Anderson, Diane K. Willimack and Richard S. Sigman U.S. Census Bureau Presented at the UNECE Statistical Data Editing Work Session, May 16-18, 2005, Ottawa, Canada

Outline Background Edit design questions Usability testing Guidelines Examples of Edits Future Directions

Background Traditional post-collection editing –Some conditions that trigger edit checks: Missing value(s) Inconsistent values Questionable value(s) Technically impossible value(s) Catch edit failures earlier: Move editing to the electronic form

How should we design the edits? 1. When should the electronic questionnaire run the edits? 2. How should the respondent be notified of an edit failure? 3. Where should the electronic questionnaire display the edits? 4. How should we phrase the edit message? 5. How should the edits look in terms of font, color, brightness?

How we answer those questions Method: Usability testing at establishments and in the usability lab –Think-aloud procedure –Observation of user behavior –Self-reported ratings of satisfaction Results: Recommendations for design changes

Preliminary guidelines 1.When should the electronic questionnaire run the edits? Guidelines: –Perform edit checks immediately. –For inter-item edits, perform those after all the items in the edit have been entered. –Check for missing data at the end. –Allow edits to be run iteratively

Immediate notification 2002 Economic Census

Running the edit before all items are complete 2001 Quarterly Financial Report

Preliminary guidelines 2. How should the respondent be notified of an edit failure? Guidelines: –Give the respondent as much control as possible. No unsolicited pop-up messages. –Avoid violating user expectations for single- purpose functions.

Respondent control: R chooses to run the edits Teacher Follow-up Survey

Unsolicited popup message 1998 Field Test Prototype for the Library Media Center Survey

Preliminary guidelines 3. Where should the electronic questionnaire display the edits? Guidelines: Clearly identify which edit failure goes with which item Provide easy navigating between an edit- failure list and the associated items.

Easy navigating between edits Survey of Industrial Research and Development (2001)

Easy navigating between edits Teacher Follow-up Survey

Clear association between edit and item Teacher Follow-up Survey

Clear association between edit and item Manufacturer’s Shipments, Inventories, & Orders Survey

Preliminary guidelines 4. How should we phrase the edit message? Guidelines: Include the item number, a description of the problem, and an action to take. Cognitively test message before fielding.

Examples of poorly worded edit messages “Check for a typing mistake” “Enter a number between 0 and 200”

Preliminary Guidelines 5. How should the edits look in terms of font, color, brightness? Guidelines: Use standard icons as expected by the business respondent

What are standard icons for edits?

Future Direction -Challenge to use guidelines -Need for experimental results -Update and add to guidelines -508 accessibility compliance -Edit design for a batch input

Questions? Contact Elizabeth Nichols Paper: pdf