Forensic Evidence in 2011: Is there a future for scientifically sound decisions in the North Carolina courtroom? Diane MB Savage, J.D. P. Michelle Fitzsimmons, Ph.D., Patent Attorney © 2011 North Carolina Attorneys for Science and Technology Durham Bar Association, October 12,

Take Away Message Some of what’s been admitted in courtrooms in NC and other states has not been based in real science Forensic Science is an oxymoron If you are going to be involved in cases that have scientific evidence, LEARN IT 2

Is there a future for scientifically sound decisions in the North Carolina courtroom? NOT IF NOTHING CHANGES! Problems that plague the forensic science community have been well understood for quite some time by scientists – not so much by the legal community. – 1998, NIJ “Forensic Fingerprint Examination Validation Studies” – 2001, The Scientific Basis for Human Bitemark Analysis – 2004, NAS “Bullet Lead Evidence” – 2004, FBI investigation of the Madrid Error 3

Call For Help Congress passed legislation in 2005 directing the National Academy of Sciences (NAS) to create an independent committee to study the forensic science community. 4

The NAS committee was composed of forensic researchers, practitioners, scientists, judges, attorneys and academics.* Committee on Identifying the needs of the Forensic Science community. Co-chaired by Harry T. Edwards and Constantine Gatsonis; 30 members Committee on Science, Technology, and Law Co-chaired by Donald Kennedy and Richard Merrell; 27 members Committee on Applied and Theoretical Statistics Chaired by Karen Kafader; 11 members * e.g. Opinion leaders from Carnegie Institute, FJC, Harvard, Johns Hopkins,NYU, NIST, Standford, UVa, U Chicago, US Federal Ct… 5

The NAS committee reviewed data from several sources. They reviewed presentations from all information submitted, including presentation from over 70 opinion leaders. – FBI, Department of Homeland Security, US Dept. Defense – U.S. Army Criminal Investigation Lab, Office of the Director of National Intelligence, NIST, DOJ, US Secret Service – NAME, SWGMAT, AAFS, AAAS, SWGFAST, SWGDDRUG, SWGGUN They sought input from an additional 25 independent reviewers with specific technical expertise. – University professors, – state offices of coroner/medical examiners, sheriffs departments, – industry consultants 6

The NAS committee heard a consistent message from forensic researchers and practitioners: “The forensic science system has serious problems that can only be addressed by a national commitment to overhaul the current structure that supports the forensic science community…This can only be done with the effective leadership at the highest levels of federal and state government, pursuant to national standards …” NAS report, p-xx, Harry T. Edwards & Constantine Gatsonis February,

NAS Assessment of Forensic Techniques “ Except for Nuclear DNA, No forensic method has been rigorously shown to have the capacity to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source.” NAS Report S-5 February,

What does this means from a scientific point of view? It is HUGE! Several forensic techniques lack the scientific underpinning to be valid, and need basic research: – Fingerprint Comparison, NAS report, p – Other Pattern Impressions, Id. p. 149 – Firearm Comparison, Id. p – Microscopic hair and fiber comparison, Id. p – Handwriting Comparison, Id. p.167 – Arson and Explosive Debris Analysis, Id. p.173 – Bloodstain Pattern Analysis, Id. p

Scientific research into these techniques will provide reliability data for the court. *Why is this evidence currently admissible? “This forensic evidence has always been admissible…” 10 TechniqueError RateSource Nuclear DNA<1% NAS, p.130 Hair Analyses* False positive= 12 % False negative = 44% NAS, p. 159 Fingerprint Analysis* No data NAS, p.141; Maryland v. Bryan Rose, No. K (MD Cir. Ct. Oct 19, 2007) Tool Marking*No dataNAS, Ballistic Imaging, 2008 p.82-3 Polygraph25%State v. Foye, 254 NC 704 (1961)

Techniques that lack scientific basis are clearly problematic, but even those techniques that are based in science have problems. 11

NAS: Scientifically sound techniques (i.e. nDNA) need rigorous examination, too: Misstating empirical data Conclusions unsupported by empirical data Conclusions that evidence originated from defendant, without looking further Dismissing contamination issues Using new tests w/o being fully validated Messaging the evidence to support the DA’s theory NAS Report p ; February,

What this means from a scientific point of view? Let’s use sports analogies For techniques that are scientifically sound, the technique must be practiced correctly: – Techniques should be validated. Validation is like defining the playing field. – Protocols should be set in stone and followed. This would be the rules of the game. – There should be no opportunity for the analyst to have discretion in the protocol. Even in football, we have the instant replay cam, which removes the discretion of the officials. 13

SWECKER REPORT of NC SBI This report raises serious issues about laboratory reporting practices from and the potential that information that was material and even favorable to the defense of criminal charges filed was withheld or misrepresented. The factors that contributed to these issues range from poorly crafted policy; lack of objectivity; the absence of clear report writing guidance; inattention to reporting methods that left too much discretion to the individual analyst; lack of transparency; and ineffective management and oversight of the Forensic Biology Section from 1987 through

What the Swecker Report Found NC-SBI has engaged in a widespread and longstanding practice of misstating the results of forensic tests, concealing evidence favorable to the defense, and withholding material and potentially exculpatory evidence from numerous criminal defendants. 230 tainted cases. (then 80 or so more) 15

Scary Numbers The rate of misreporting was nearly 25%: 230 problem cases of the 932 identified. Lab notes reflected that a positive presumptive test for the presence of blood was followed by a confirmatory test that yielded results that were ‘negative,’ inconclusive’ or ‘no result,’ but did not include this information in the final report.” Didn’t report cases where “slight positive” result. 16

Swecker’s Recommendations Immediate notification to the appropriate District Attorney to review the listed cases and notify any convicted defendants who may have been adversely impacted; A legal review of the Lab’s reporting methods; Enhanced training; More transparency in the Lab’s policies and procedures; Designation of an Ombudsman position to review and quickly act on information regarding potential laboratory issues and errors. 17

Neufeld’s Study: Why folks found guilty were later exonerated with DNA? 72 analysts 52 laboratories 25 States Defense rarely cross examined Judges seldom provided relief 18

Post-mortems of DNA Exonerations [Neufeld] * Courtesy of Michael Saks, Jennifer Friedman, Barry Scheck 19

Forensic Labs in General Staffed by analysts who by their own words are sworn law enforcement officers first; Support the DA’s theory; Tend not to talk to defense atty w/o permission of DA; Focus on person charged, especially in a case where the evidence is not clear cut; Generally adversarial; 20

How “Cheating” Occurs Hiding evidence and not reporting results; Testifying about results even though outdated chemicals were used; Reporting results in a way to confuse; Non probative evidence presented as probative; Exculpatory evidence discounted; 21

MORE Even when you ask for the basis of their opinion, they have nothing to support the testimony e.g. journal articles, texts etc; Can’t point to publications in peer-reviewed journals so another expert could review and question contention and/or repeat findings; Lack of mandatory standardization; Forensic science culture v. scientific culture. 22

MORE Inaccurate frequency or statistics presented; Statistics provided without empirical support; Non-numerical statements provided without empirical support; Little to no pictorial documentation to support contentions; Making statements that go beyond established facts; Changing protocols to match testimony. 23

In science labs, this is avoided by adopting a “quality systems” approach. There is no discretion regarding analyst reporting and everything is documented; Validation Protocols & Reports – Instrument Qualification Is it properly designed? (DQ) Is it installed and has everything it needs? (IQ) Does it operate as designed? (OQ) Does it perform reproducibly per pre- approved specifications? (PQ) – Method Validation- Is the analytical method acceptable for it’s intended propose? SpecificityLinearity AccuracyPrecision RangeDetection limit Quantization limitrobustness 24 Method Change Control Discrepancy and failure investigation Reprocess/ rework Annual record review Stability testing Training Senior Management Oversight Why doesn’t the Forensic community adopt the same policy?

Can the forensic industry adopt a research atmosphere? Commitment to “Truth seeking” Scientist mindset, not law enforcement: Derive the theory from the data instead of the data conformed to the theory. Careful experimental design and attention to detail: Minimize the possibility of unknown variables. Truly validated protocols Careful conclusions: Conclude only what the data says; no stretching or speculating. Work for public safety, not the prosecution or the defense. 25

For nearly a century, courts have begun with the presumption that forensic evidence is valid. “Bad techniques” with little or no scientific underpinnings are admissible. Admissible because “it has always been admissible” “Good techniques” ( ie DNA) with poor execution of the scientific method are admissible. Courts have not required labs to perform the tests accurately, as they assume a “good technique” is not susceptible to manipulation. 26

JUSTICE DEMANDS GOOD SCIENCE Courts cannot now blindly follow precedent that rests on unfounded scientific premises. This will lead to unjust results. Not to mention financial costs! 27

Change is coming ? The Federal Judicial Manual on Scientific Evidence, 3 rd Ed. was released on 9/28/2011 and contains guidance for Federal and State Judges for assessing, among other things, Forensic Evidence and DNA. avgeneral?openpage&url_r=/public/home.nsf/pages/1448 NAS has created four national “Forensic Science Centers of Excellence” with substantial federal funding. Federal and State Judges are beginning to exercise their gatekeeping role when reviewing forensic evidence. Will it be coming to North Carolina? 28

Hon. Harry T. Edwards Little research to confirm the validity and reliability of forensic disciplines Little research programs on human observer bias and sources of human error in forensic examinations Lack of autonomy of crime laboratories Absence of rigorous mandatory certification requirements for practitioners Absence of uniform mandatory accreditation programs for labs Failure to adhere to robust performance standards Failure of forensic experts to use standard terminology in reporting on and testifying about the results of forensic science investigations Lack of effective oversight Gross shortage of adequate training and continuing education of practitioners Presentation at the Superior Court of the District of Columbia Conference on: the role of the courts in an age of developing science and technology 5/6/

Judge Gertner The National Academy of Sciences’ call for change in forensic sciences will not be successful until lawyers fairly bring these standards to the attention of the courts, and the judges, both district and appellate, rigorously enforce them. COMMENTARY ON THE NEED FOR A RESEARCH CULTURE IN THE FORENSIC SCIENCES, 58 UCLA Law Review 789 (2011) 30

Judge Howard’s Order Falsely portrayed to the jury that he conducted a test for blood which indicated blood, not some substance which might be blood; The prosecutor reinforced the misleading testimony by arguing during closing in listing the evidence of petitioner's guilt for the jury to "think about the blood on the boots; His misleading testimony is imputed to the State; Agreed with the attorneys' characterization of the substance as blood without clarifying the limits of the test. Goode v. Branker, Order p

JUSTICE SCALIA AND THE NAS REPORT “nor is it evident that what respondent calls neutral scientific testing is as neutral or as reliable as respondent suggests” “because forensic scientists often are driven ….by a need to answer a particular question related to …a particular case, they sometimes face pressure to sacrifice appropriate methodology for the sake of expediency.” Melendez-Dias v. Massachusetts, 129 S.Ct 2527 (2009) 32

SMOKE AND MIRRORS Not problems of the 80s and 90s Won’t get better by having a lawyer review requests first Can’t have a judge running a lab Problems still remain 33

Questions to ask Is it an ethical violation for prosecutors to offer evidence with a statement of individualization given the NAS report and the Swecker report? Are we as defense attorneys ineffective if we do not have the knowledge necessary to effectively litigate the scientific issues in the case? 35

IN SCIENCE IF IT LOOKS LIKE A DUCK AND QUACKS LIKE A DUCK YOU MUST PROVE IT IS A DUCK 36

NAS RECOMMENDATIONS Remove all public forensic labs and facilities from the control of law enforcement agencies or prosecution offices Establish independent federal entity National Institute of FS Purpose - independent accreditation and oversight of forensic scientists and forensic labs 37

CLOSINGS 2008 – Manufacturing Plant Closes after Pet Food recalled – Pedigree issues warnings; Virus that forced Genzyme plant closing contained to Allston - Framingham, MA; 2010 – Preliminary report finds 230 tainted cases at NC SBI lab and lab remains open; Jan. 8, 2011 – Germany shuts 4700 Farms in Dioxin scare; Jan. 9, 2011 – Duke trial based on bad science – trial shut down; doctor padded resume and resigns; May 31, 2011 The Consumer Product Safety Commission has announced a recall of 31,100 bicycle helmets in the U.S. and about 2,500 in Canada imported by Bell Sports. The agency reported that the buckles on the chin strap may detach, causing the helmet to fall off suddenly. There has been one report of the buckles detaching during an accident, leading to an injury that required stitches. October 12, 2011 – SBI lab still operating; one analyst fired. 38