1. Juror Understanding of Random Match Probabilities Dale A. Nance Case Western Reserve University August, 2007

2. Focus of Presentation What we know about how jurors react to testimony reporting a match between the defendant and the perpetrator and presenting a random match probability (RMP) Experiments assessing juror reactions

3. Eight Common Hypotheses About Cognitive Error by Jurors 1. The Prosecutors Fallacy 2. Neglect of Lab Error 3. Improper Combination Strategies 4. Vividness 5. Defense Attorneys Fallacy 6. Defense Attorneys (Extreme) Fallacy 7. The Inversion Fallacy 8. Misaggregation

4. 1. The Prosecutors Fallacy The chance of a coincidental match with an innocent man is 1 in 40,000. What the Expert Says

5. 1. The Prosecutors Fallacy The chance of a coincidental match with an innocent man is 1 in 40,000. What the Expert Says The chance that the accused in innocent is 1 in 40,000, so the odds that he is guilty must be 39,999 to 1. What the Jurors Think

6. 2. Neglect of Lab Error The chance of a coincidental match with an innocent man is 1 in 40,000. What the Expert Says

7. 2. Neglect of Lab Error The chance of a coincidental match with an innocent man is 1 in 40,000. What the Expert Says The chance that the accused, though innocent, would be implicated by either coincidence or lab error is 1 in 40,000. What the Jurors Think

8. 3. Combination Errors (Averaging) The chance of a coincidental match with an innocent man is 1 in 40,000. The chance of a false positive lab error is about 1 in 1,000. What the Expert Says

9. 3. Combination Errors (Averaging) The chance of a coincidental match with an innocent man is 1 in 40,000. The chance of a false positive lab error is about 1 in 1,000. What the Expert Says The chance that the accused, though innocent, would be implicated by a coincidental match or lab error is 1 in 20,500. What the Jurors Think

10. 4. The Vividness Hypothesis The chance of a coincidental match with an innocent man is one in a billion. What the Expert Says

11. 4. The Vividness Hypothesis The chance of a coincidental match with an innocent man is one in a billion. What the Expert Says One in a billion! Thats all I need to know. Hang the bastard! What the Jurors Think

12. 5. The Defense Attorneys Fallacy The chance of a coincidental match with an innocent man is 1 in 40,000. Yes, out of 12,000,000 adult men, about 300 will match. What the Expert Says

13. 5. The Defense Attorneys Fallacy The chance of a coincidental match with an innocent man is 1 in 40,000. Yes, out of 12,000,000 adult men, about 300 will match. What the Expert Says If 300 men will match, then this DNA evidence tells us nothing. I should just decide the case on the eyewitness evidence. What the Jurors Think

14. 6. The Defense Attorneys (Extreme) Fallacy The chance of a coincidental match with an innocent man is 1 in 40,000. Yes, out of 12,000,000 adult men, about 300 will match. What the Expert Says

15. 6. The Defense Attorneys (Extreme) Fallacy The chance of a coincidental match with an innocent man is 1 in 40,000. Yes, out of 12,000,000 adult men, about 300 will match. What the Expert Says If 300 men will match, then the chance the accused is guilty must be only 1 in 300. What the Jurors Think

16. 7. The Inversion Fallacy The chance of a coincidental match with an innocent man is 1 in 40,000. What the Expert Says

17. 7. The Inversion Fallacy The chance of a coincidental match with an innocent man is 1 in 40,000. What the Expert Says The chance that the accused in guilty is just 1 in 40,000. This prosecutor must be from Durham. \ What the Jurors Think

18. 8. Misaggregation The chance of a coincidental match with an innocent man is 1 in 40,000. What the Expert Says

19. 8. Misaggregation The chance of a coincidental match with an innocent man is 1 in 40,000. What the Expert Says Without the DNA evidence, I would place the odds of guilt at 2:1 against. With this DNA evidence, the odds of guilt are about 2:1 for. What the Jurors Think

20. 8. Misaggregation: How Bad Is It? For a RMP = 1 in 40,000, and considering only the chance of: Coincidental match, posterior odds should be 40,000 times the prior odds: Coincidental match or lab error (at a rate of 1 in 1,000), posterior odds should be about 1000 times the prior: Coincidental match, lab error, or other sources of error (like police planting of evidence), assessed by the average juror at about 1 in 50, the posterior should be about 40 times the prior: PRIOR POST. ODDS ODDS 1 :2 20,000:1 1:2 500:1 1:2 20:1

21. 8. Misaggregation: What Can Be Done About it? 1. Give RMP testimony in the form of probabilities focused on the defendant, rather than frequencies focused on the population: –The probability that defendant would match if he were innocent is 1 in 40,000. rather than –1 in 40,000 people in the population share this DNA profile.

22. 8. Misaggregation: What Can Be Done About it? 2. Give testimony explaining the RMP by showing results of hypothetical Bayes Rule calculations. For example, with RMP= 1 in 40,000 and ignoring other sources of error: Prior Probability Posterior Probability 1/10 of 1% 97.56% 1% 99.75% 20% 99.99% 50% 99.99% 70% 99.99%

23. 8. Misaggregation: What Can Be Done About it? Incorporating information about lab error rates into the calculation produces lower posterior probabilities: Prior Prob. Post. Prob. Post. Prob. (ignoring lab error) (incorp. lab error) 1/10 of 1% 97.56% 49.42% 1% 99.75% 90.79% 20% 99.99% 99.59% 50% 99.99% 99.90% 70% 99.99% 99.96%

24. Conclusions Pro-prosecution fallacies: extant but correctible by argument or by restrictions on form of RMP presentation Pro-defense fallacies: extant but of declining importance as RMP becomes very small Pro-defense error (misaggregation): serious but potentially amenable to Bayesian instruction