1. Human Factors in Voting Systems John O’Hara IEEE Usability-Accessibility Working Group Chair HFES Voting System Task Force Chair Advisory Board Meeting Washington, DC October 16, 2002

2. Topics Background Technical perspective The voting system Human factors engineering of voting systems Recent HFE standards activities for voting systems Beyond the current standards

3. Background Voting technology is changing FEC revised its voting system standards to accommodate new technology FEC’s standards do not completely address human factors aspects of voting systems IEEE standard will strengthen the treatment of usability and accessibility FEC is sponsoring R&D to more completely address voting system human factors and usability

4. Voting Technology Trends DRE and optical scan only technologies increasing in usage from IEEE Spectrum, 2000

5. Technical Perspective Complex systems –socio-technological systems that integrate hardware, software, and human resources to achieve a specific goal –high-risk, high reliability –often serve critical infrastructure missions, such as defense/security, transportation, health care, and energy

6. Technical Perspective Performance is addressed through systems engineering and defense-in-depth design –high design standards –redundancy and diversity for system elements –analysis of likely failures and design for recovery from them –personnel handle normal and failure conditions through procedures and training

7. How is Human Performance Addressed? Human performance is addressed through human factors engineering (HFE) Knowledge about human characteristics and capabilities is developed in many scientific disciplines –physiology, psychology, and sociology/anthropology Human factors is: – Research specifically devoted to developing knowledge about the integration of human and machine systems – Application of that knowledge to the design of systems and equipment

8. General Human Factors Activities Goals –highly reliable system and human performance –simplicity and standardization –error tolerance (minimization, recognition, and recovery) Requirements analysis (tasks, users, etc.) Detailed design of equipment –design characteristics and functions –application of HFE guidelines Development of supporting infrastructure –instructions, procedures, and training Test and evaluation Performance monitoring and feedback

9. Is Systems Engineering an Appropriate Model for Voting Systems? Voting systems involve the integration and coordination of hardware, software, and human resources to achieve a critical mission The system elements all contribute to mission success although they are “loosely coupled” and distributed –e.g., good ballot design software does not guarantee that good ballots will be designed by election officials using the software Human performance is vital to success of the mission Yes

10. The Voting System from Draft IEEE P-1581 System elements are the infrastructure that accomplish the voting process

11. The Voting System Election Management System –administrative database –ballot generation –election programming –ballot validation Control Subsystem –equipment preparation –pre-delivery testing –tests at the polling place –opening the polling place –enabling a ballot –error recovery –closing the polling place –polling place reports Vote Recording Subsystem –precinct voting –on-site absentee voting, early voting, and state-wide voting –remote absentee voting Reporting Subsystem –polling place reports –precinct reports –consolidated reports –system audit log –access to election data

12. User Groups Vendors Voters Test Labs Voting Officials Precinct Workers Maintenance Personnel

13. Lesson Learned from the Analysis of Complex System Failure Overall performance is a function of the integrated system –failure is often the result of the interaction of system elements –difficult to focus on one component of the system Quick fix is often to throw technology at a problem –the technology “halo” effect –technology itself does not enhance or detract from human performance –often old problems are solved, but new problems are created

14. Broader and Unanticipated Effects of Technology Changes in the roles and responsibilities of people working in the system Changes in the ways people accomplish their tasks Changes in the organizational structure –e.g., need for an Information Technology Department to manage software configuration, etc. Digital technology evolves much more quickly than other technologies and organizations need to be able to respond

15. Voting System Failure Since voting system elements are “loosely coupled,” standards aimed at one aspect of the system may not guarantee high- reliability of the whole system A comprehensive, hierarchical approach to voting system human factors is needed –individual standards addressing system elements (and probably different user groups) –an overall standard or guidance document that ties the individual elements together

16. Supporting Human Performance Through Human Factors Thorough analysis of human tasks –for all significant user groups in the system –address primary and secondary tasks Human-centered equipment design –enable user to devote full attention to the task at hand –to minimize secondary tasks (minimize attention shifts) –make the equipment as transparent as possible Some key principles –design reflects basic human cognitive and physiological capabilities –positive guidance for appropriate task behavior –high-degree of simplicity, standardization, and consistency –error tolerance (minimize error through design and allow errors to be recognized and recovered from)

17. Supporting Human Performance in the System Comprehensive test and evaluations –“usability tests” to discover and address unanticipated issues –verifications against human factors design standards –validation that desired system performance criteria are met Well-designed procedures and instructions –all user groups –for all user tasks Training, familiarization, and help Defense-in-depth system design –error tolerance across all system elements –multiple barriers establish layers of defense against errors and failures

18. VSS Usability Appendix Provided a systems engineering view of the voting system usability Guidance on key aspects of voting system human factors engineering was provided –equipment design –infrastructure support, such as familiarization and training Fairly high-level and narrow focus

19. IEEE P-1583 Usability-Accessibility Sections Builds on the VSS Usability Appendix Focus –equipment design requirements –voter interface –standards verification for compliance

20. Topics Addressed 6.5Usability and Accessibility –6.5.1General Principles –6.5.2Overall Design and Layout of the Voting Location –6.5.3Voter Identification and Authentication –6.5.4Ballot and Information Presentation –6.5.5Voter Input and Response –6.5.6Navigation and Interaction with Ballots –6.5.7Preventing and Minimizing Voter Errors –6.5.8Help and System Failure –6.5.9Voter Familiarization and Training 7.5Usability and Accessibility Testing

21. Further IEEE Standard Development Increase level of detail for voting equipment human-system interfaces –touch screens –cursor control devices and behavior Improve accessibility standards Develop standards verification criteria

22. Beyond the Current Standards Expanded the scope of the standards to the whole voting system –treatment of other technological approaches to voting equipment –test and evaluation methods beyond verification –ballot preparation and testing –voter familiarization and training –equipment setup General guidance on “how to” –we have standards, e.g., for equipment design, but do not offer guidance to designers on how (methods) to accomplish good HFE Tie it all together –standards and guidance should be presented as an integrated whole

23. Beyond the Current Standards VendorsTest Labs Precinct Workers Voting Officials Testing Handbook Testing Handbook Ballot Design Handbook Ballot Design Handbook Equipment Design Handbook Equipment Design Handbook Ballot Design Standard Ballot Design Standard Equipment Standard Equipment Standard Voting System HFE Voting System HFE Testing Standard Testing Standard Voter Training Handbook Voter Training Handbook Voter Training Standard Voter Training Standard Vendors