1. Designing Interactive Edits for U.S. Electronic Economic Surveys and Censuses: Issues and Guidelines Elizabeth M. Nichols, Elizabeth D. Murphy, Amy E. Anderson, Diane K. Willimack and Richard S. Sigman U.S. Census Bureau Presented at the UNECE Statistical Data Editing Work Session, May 16-18, 2005, Ottawa, Canada

2. Outline Background Edit design questions Usability testing Guidelines Examples of Edits Future Directions

3. Background Traditional post-collection editing –Some conditions that trigger edit checks: Missing value(s) Inconsistent values Questionable value(s) Technically impossible value(s) Catch edit failures earlier: Move editing to the electronic form

4. How should we design the edits? 1. When should the electronic questionnaire run the edits? 2. How should the respondent be notified of an edit failure? 3. Where should the electronic questionnaire display the edits? 4. How should we phrase the edit message? 5. How should the edits look in terms of font, color, brightness?

5. How we answer those questions Method: Usability testing at establishments and in the usability lab –Think-aloud procedure –Observation of user behavior –Self-reported ratings of satisfaction Results: Recommendations for design changes

6. Preliminary guidelines 1.When should the electronic questionnaire run the edits? Guidelines: –Perform edit checks immediately. –For inter-item edits, perform those after all the items in the edit have been entered. –Check for missing data at the end. –Allow edits to be run iteratively

7. Immediate notification 2002 Economic Census

9. Running the edit before all items are complete 2001 Quarterly Financial Report

11. Preliminary guidelines 2. How should the respondent be notified of an edit failure? Guidelines: –Give the respondent as much control as possible. No unsolicited pop-up messages. –Avoid violating user expectations for single- purpose functions.

12. Respondent control: R chooses to run the edits 2004-05 Teacher Follow-up Survey

13. Unsolicited popup message 1998 Field Test Prototype for the Library Media Center Survey

14. Preliminary guidelines 3. Where should the electronic questionnaire display the edits? Guidelines: Clearly identify which edit failure goes with which item Provide easy navigating between an edit- failure list and the associated items.

15. Easy navigating between edits Survey of Industrial Research and Development (2001)

16. Easy navigating between edits 2004-05 Teacher Follow-up Survey

17. Clear association between edit and item 2004-05 Teacher Follow-up Survey

18. Clear association between edit and item Manufacturer’s Shipments, Inventories, & Orders Survey

19. Preliminary guidelines 4. How should we phrase the edit message? Guidelines: Include the item number, a description of the problem, and an action to take. Cognitively test message before fielding.

20. Examples of poorly worded edit messages “Check for a typing mistake” “Enter a number between 0 and 200”

21. Preliminary Guidelines 5. How should the edits look in terms of font, color, brightness? Guidelines: Use standard icons as expected by the business respondent

22. What are standard icons for edits?

23. Future Direction -Challenge to use guidelines -Need for experimental results -Update and add to guidelines -508 accessibility compliance -Edit design for a batch input

24. Questions? Contact Elizabeth Nichols elizabeth.may.nichols@census.gov Paper: http://www.census.gov/srd/papers/pdf/rsm2 005-03.pdf