1. Audit Purpose of Audit Quality assurance procedure Check accuracy of machine tally of ballots Ballots for a contest are sampled, manually verified, and results checked against vote tabulation Boulder County’s Audit Risk-minimizing approach – minimize risk of confirming an incorrect election outcome Meets “higher level of statistical confidence” part of CRS 1-7-514 Selecting contests – probability proportional to the inverse of the contests’ margin. Close contests more likely to be selected. Selecting Ballots – statistical algorithm which uses the contest’s margin, weighting by the number of times the contest appears in a batch, and random selection. Uncontested contests ignored

2. Audit Contest & Batch Selection MBB 3 Tally …. MBB 2 MBB 1 Batch Reporting Zero Cum 1 Cum 2 … Cum n Contest SelectionBatch Selection Weight contests by relative MarginCreate Individual MBB Reports M margin V number of participating voters Weight batches by # times the contest appears in the batch Randomly select

3. Contest Selection Example Ballots Cast Abs Margin Rel Margin (% margin) 1/%marginWeightCumulative Weight Contest 1 100001010/10000 = 0.001 10001000/1150 = 0.870 0.8700 Contest 2 10000100100/10000 = 0.01 100100/1150 = 0.087 0.9570 Contest 3 10000200200/10000 = 0.02 5050/1150 = 0.043 1.0000 11501.0 0 0.8700 0.95701.0 Contest 1 C 2 C 3

4. Contest Selection Caveats Select contests one at a time until the audit budget has been reached. Contests having unopposed candidates (or no candidates) will not be audited. Contests involved in a recount will be ignored. Their results will be audited after the recount.

5. Batch Selection Algorithm Used – Negative Exponential (NEGEXP) Proposed in 2008 by Aslam, Popa, and Rivest Ballot batches selected for audit with probability based on the number of times the contest appears in the batch. Number of batches to audit determined by the desired confidence that any outcome-impacting error, if present, would be found. Better operational efficiency in large elections. Requires a large number of random numbers  use random number generator Additional Constraints Additional batches added to ensure all scanners are audited at least once. Canvass board may nominate batches to be audited.

6. SUNMONTUESWEDTHURFRISAT 6/24 Election Day 6/25 Provisionals 6/26 Provisionals 6/27 Provisionals 2PM Audit Kick- off 6/28 6/29 9:30 am Audit batch pull 6/30 9:30 am Audit 7/1 9:30 am Audit 2PM Initial Data Review 7/2 Provisionals 8-day Ballot processing 9:30 am Audit 7/3 9:30 am Audit if needed 3PM Data Q&A and final Election Data Provided 7/4 4 th of July 7/5 7/67/7 2PM Final Canvass Board meeting 7/87/97/107/117/12 7/137/147/157/167/177/187/19 2014 Primary Election Canvass Reconciliation & Audit Schedule