1. Improving Forensic Science Through Laboratory Experiment (NSF Award Number 0622477) Dr. Roger G. Koppl Institute for Forensic Science Administration Dr. Lawrence Kobilinsky John Jay College of Criminal Justice Dr. Robert Kurzban University of Pennsylvania

2. Summary The experimental method is central to science It may be applied to the organization of forensics Larry Miller’s pioneering study illustrates We are in the Miller tradition We find “strategic redundancy” may help reduce the error rate in forensic science

3. The Problem Context Errors are bad The forensic science community has come under fire We do not yet have an effective strategy of response to recent challenges Our experiments in may help

4. Two (bad) models of the forensic scientist

7. Our Model

9. If forensic scientists are human beings and not cartoon characters, then well-established experimental techniques apply

10. Example: The Miller Study Miller, Larry S. “Procedural Bias in Forensic Science Examinations of Human Hair,” Law and Human Behavior, 1987, 11: 157-163. One known hair per questioned hair vs. five known hairs per questioned hair Evidence lineup for traditional hair microscopy reduced rate of false positives from 30.4% to 3.8%

11. Lessons From the Miller Study Evidence lineups may increase reliability of forensic evidence Experimental methods may help us improve the organization of forensic science

12. Institutional Structure of Forensics Evidence is typically examined by one lab only Sometimes evidence should be sent to more than one crime lab That’s “strategic redundancy”

13. No Strategic Redundancy

14. Strategic Redundancy

15. In our experiments... “Senders” represent forensics laboratories “Receivers” represent judge or jury (trier of fact) Sender(s) looks at some evidence and give a “report” to the Receiver Receiver guesses at the truth One Sender vs. Three Senders

16. Basics of Experimental Design I Sender(s) evaluate evidence and give a report to a Receiver Sender(s) shown one of three shapes: Receiver guesses which shape was shown

17. Basics of Experimental Design II Receiver paid $5.00 for correct guesses, $2.00 for incorrect guesses Senders are paid $3.00 for inducing correct guesses Senders are also given a payment of $1.00 or $5.00 to make a report that may be inaccurate

18. What the Sender Sees

20. What the Sender Fills Out

22. One-Sender Treatment Subjects paired into Senders and Receivers Senders informed of “correct shape,” “supplementary shape,” the value of supplementary shape $1.00 for the supplementary shape gives Senders an incentive to issue an accurate report $5.00 value gives Senders an incentive to issue an inaccurate report -- if the supplementary shape is not the correct shape

23. Three-Sender Treatment Each Receiver is matched with 3 Senders Otherwise identical to one-Sender treatment, including payoffs Will Senders report the correct shape more often? Will Receivers guess the correct shape more often?

24. Results: Errors in Messages Fell in three-sender treatment 1-Sender: 86% 3-Sender: 67% ( χ 2 =5,d.f.=1 p = 0.02) $5.00 cases; supplementary  correct

25. Results: Systemic Error Rate Fell under dramatically democratic epistemics 1-Sender: 75% 3-Sender: 25% ( χ 2 =20,d.f.=1 p < 0.0005) Supplementary  correct; at least one Sender gets $5.00 for supplementary shape 36% vs. 15% for all cases 47% vs. 21% if supplementary  correct

26. Why systemic error rates fall by more Assume one Sender never reports correct shape, two always report correct shape Error rate in messages = 33%, Systemic error rate = 0 Effect of strategic redundancy

27. Redundancy Our experiments suggest the value of backup systems Emergency lights in large buildings Spare tires in our automobile trunks Improved backup systems in forensics Experiments can help find them

28. So what? Strategic redundancy may help reduce error rates in forensic science More experiments should be run –Repeated iterations with reputation –Uncertainty and Ambiguity Laboratory experiments in the organization of forensic science may help us reduce error rates in forensic science