12/9-10/2009 TGDC Meeting TGDC Recommendations Research as requested by the EAC John P. Wack National Institute of Standards and Technology

12/9-10/2009 TGDC Meeting Additional research EAC requested that NIST conduct TGDC Recommendations-related research Six items requested as response to EAC Standards Board and Board of Advisors resolutions from Dec 2007 meetings that focused on the Recommendations Report available at research-areas.pdf Page 2

12/9-10/2009 TGDC Meeting Overview of six items Alternatives to software independence Developing standards for ballot on demand systems Impact of the VVSG on vote by phone systems Ramifications of separately testing and certifying components plus requirements for interoperability Impact of the VVSG on early voting or vote centers Developing alternatives to goal level requirements in the VVSG Page 3

12/9-10/2009 TGDC Meeting Area 1: Alternatives to Software Independence Page 4 Should retain focus on security, verifiability, and auditability in the VVSG Research should be conducted on what changes need to be made to the VVSG to accommodate each alternative

12/9-10/2009 TGDC Meeting What is Software Independence Accuracy of the election doesn’t depend on the voting system software working without error Focuses on the accuracy of the electronic ballot records In practical terms, audits of electronic records can be performed Systems today need an audit record that is voter- verifiable and semi-permanent, e.g., paper records Page 5

12/9-10/2009 TGDC Meeting Current analysis To retain focus on security, alternatives to SI have significant ramifications: Increased design and development costs Likely need for greater testing Modifications to the VVSG will require several years to develop: Will require significant research Will require standardization efforts Need for funding or possible development incentives Ramifications to other areas of the VVSG that will need further study, e.g., usability, accessibility Page 6

12/9-10/2009 TGDC Meeting Area 2: Ballot on demand BOD: device that can print ballots on demand for use in elections Reduces need to pre-print and transport ballots to polling place NIST to research the feasibility of including BOD requirements in the VVSG Do election official needs require further research before requirements can be written? Page 7

12/9-10/2009 TGDC Meeting Area 3: Vote by phone systems VBP: voters use telephones and guided prompts to place votes, electronic and paper ballots printed out at remote office Does the VVSG, as written, prohibit VBP? Page 8

12/9-10/2009 TGDC Meeting Area 4: Separately certifying components & interoperability Develop a feasibility study of the ramifications of the EAC separately testing and certifying components Research requirements for interoperability between systems and system components NIST to research whether a specific standard for format of electronic election data can be required Page 9

12/9-10/2009 TGDC Meeting Testing & Certifying Components Change in philosophy, and potentially affects: VVSG (e.g., classes, requirements, etc) EAC procedures Test Lab processes, tests, reporting Issues to consider, include: Which components? (e.g., all?, only those that plug & play?) Need new requirements for integrating components into the voting system Define voting system architecture, interfaces, protocols, data formats, etc. Potentially restrictive to manufacturers, require hardware design changes How do you ensure an integrated system works correctly and conforms to the VVSG? Can’t test all aspects of voting system unless all components are configured Need new test methods to test for interoperability Still need to test the voting system as a whole, i.e., “The system is more than the sum of its parts.” Page 10

12/9-10/2009 TGDC Meeting Interoperability Need to define what would be interoperable? Devices? (e.g., vote capture, tabulator) Data? (e.g., Ballot configurations, election results) VVSG conformance testing does not address interoperability: Conformance to a standard is necessary, but not sufficient, implementations may still differ A separate interoperability testing program would need to be contemplated Interoperability testing is between 2+ items Can be expensive and time consuming Page 11

12/9-10/2009 TGDC Meeting Standardized Data Format Current VVSG requirement = use standardized format Allows manufacturer to determine format Translators can make data exchangeable Selecting a specific standardized data format Must support all voting variations Should be vetted by U.S. voting system manufacturers Should be simple and unambiguous to implement Should first demonstrate that it can enable interoperability (i.e., achieve the objective for a standardized format) Format standard must be explicit with respect to data structures, data definitions, allowable data values, etc. Page 12

12/9-10/2009 TGDC Meeting Area 5: Early voting & vote centers Addresses questions as to whether VVSG prohibits or impacts use of voting equipment in early voting or vote centers Especially of concern with regard to use of epollbooks Page 13

12/9-10/2009 TGDC Meeting No impacts seen, CRT requirements accommodate early voting and vote centers VVPAT requirements anticipated multi-precinct use of equipment Electronic poll book requirements permit network connections to central voter registration databases Caveat: cannot network poll books to vote capture devices and databases at same time Current analysis Page 14

12/9-10/2009 TGDC Meeting Page 15 Area 6: Goal Requirements A requirement that purposely may be broad and less precise as opposed to being very specific. States and requires a desired performance or behavior, but does not specifically state how that performance or behavior is to be met. Goal requirements are not necessarily good or bad; they are used for specific purposes.

12/9-10/2009 TGDC Meeting Goal requirements Concerns expressed with goal requirements They may be untestable, ambiguous, testing may not be repeatable If they are “should” requirements, there is a question as to whether they are required Page 16

12/9-10/2009 TGDC Meeting Page 17 Why Goal Requirements? Often done to avoid constraining design Some express a goal to be met by the manufacturer but specific requirements cannot yet be developed Some are performance requirements in which more specific requirements may limit innovation Can be repeatable according to a test protocol

12/9-10/2009 TGDC Meeting Some could be deleted or included as informative text or as “should” requirements, e.g., integratability Some are fine as long as tests are included, e.g., usability Current analysis Page 18

12/9-10/2009 TGDC Meeting Presentations to follow… Alternatives to SI and End-to-End Workshop Report – Nelson Hastings Auditing Concepts, Ballot on Demand, Vote by Phone – David Flater Common Data Format Workshop Report – Ben Long Accessibility, Usability Research Update – Sharon Laskowski Page 19