1. Fundamentals of Forensic DNA Typing Slides prepared by John M. Butler June 2009 Chapter 13 Quality Assurance

2. Chapter 13 – Quality Assurance Chapter Summary DNA testing is a multi-step, technical process that needs to be performed by qualified and effectively trained personnel to ensure that accurate results are obtained and interpreted correctly. Community-wide quality assurance standards, laboratory accreditation and audits, regular analyst proficiency tests and on- going continuing education, method and instrument validation, standard operating procedures, adequate controls and reference materials along with secondary reviews by other qualified analysts all act as checks and controls on DNA results produced in forensic laboratories. These quality assurance measures help maintain good laboratory practice and produce accurate scientific results. Casework management systems (laboratory information management systems, LIMS) along with robotics can further aid production of quality results in a timely fashion.

3. Brief Historical Overview Profiles in DNA (Sept 1999) 3(2): 10-11 Quality problems in late 1980s with DNA testing TWGDAM established under FBI Lab sponsorship in 1988 NRC I (1992) and NRC II (1996) issued reports recommending formal QA programs DNA Identification Act of 1994 lead to formation of DNA Advisory Board (DAB) DAB Standards issued in Oct 1998 and Apr 1999 When DAB was dissolved in 2000, SWGDAM assumed leadership role NIST had membership on the DNA Advisory Board and actively participates in SWGDAM

4. Inspections/ Audits ASCLD-LAB Accreditation DAB Standards- SWGDAM Guidelines Validated Methods (using standards and controls) Proficiency Testing of Analysts Ensuring Accurate Forensic DNA Results

5. FBI Quality Assurance Standards SWGDAM Guidelines Laboratory-specific Standard Operating Procedures and Interpretation Guidelines Individual Analyst Practice Each Case Report Case Review Validation (Reference Materials) Audits NDIS Procedures Training, Proficiency Tests, and Continuing Education Individual Analyst Practice Each Case Report Instrument Accreditation If court presentation… Defense Review

6. DNA Profile Comparison QK Court Database Search Exclusion (no match) Inclusion (match) May match another (K’) Collection Extraction Quantitation STR Markers Data Interpretation Sample Storage Amplification Statistical Interpretation Characterization Separation/ Detection Evidence (Question) sample “Q” Biology Technology Genetics Serology Profile put on database Steps Involved Plea Report (with statistical weight) Q = K Q ≠ K Crime committed Biological material transferred Collection Extraction Quantitation STR Markers Data Interpretation Sample Storage Amplification Separation/ Detection Reference (Known) sample “K” Profile put on database Steps Involved Suspect developed QUALITYASSURANCEQUALITYASSURANCE QUALITYASSURANCEQUALITYASSURANCE May be Inconclusive due to Forensic Issues (degradation, mixtures, low levels) May be Inconclusive due to Lack of Available Reference

7. Scientific Working Group on DNA Analysis Methods (SWGDAM) Organized originally by FBI Laboratory as Technical Working Group on DNA Analysis Methods (TWGDAM) in 1988 Meets semiannually – each January and July Organized into eight subcommittees: –Quality Assurance, CODIS, mtDNA, Mass Disasters/Missing Persons, Expert Systems, Serology, Y-STRs, and Mixture Interpretation Membership (usually ~50 attend) from public forensic DNA laboratories around the U.S.

8. Organizations Aiding Forensic DNA Standardization European DNA Profiling Group (EDNAP) –Working group of International Society of Forensic Genetics (ISFG) –Examine technologies and run interlab studies –28 participants from 19 different countries European Network of Forensic Science Institutes (ENFSI) –Defines policy within European Union –ENFSI DNA Working Group equivalent of SWGDAM –85 participants from 32 different countries Have challenges with language differences due to many countries involved http://www.isfg.org/ ednap/ednap.htm http://www.enfsi.eu/ http://www.isfg.org/ The NIST Human Identity Project Team participates in EDNAP, ENFSI, and ISFG.

10. DNA Identification Act (1994) 42 § 14131. Quality assurance and proficiency testing standards (a)Publication of quality assurance and proficiency testing standards (1) (A) Not later than 180 days after September 13, 1994, the Director of the Federal Bureau of Investigation shall appoint an advisory board on DNA quality assurance methods from among nominations proposed by the head of the National Academy of Sciences and professional societies of crime laboratory officials. (B) The advisory board shall include as members scientists from State, local, and private forensic laboratories, molecular geneticists and population geneticists not affiliated with a forensic laboratory, and a representative from the National Institute of Standards and Technology. (C) The advisory board shall develop, and if appropriate, periodically revise, recommended standards for quality assurance, including standards for testing the proficiency of forensic laboratories, and forensic analysts, in conducting analyses of DNA. Public Law 103-322 DNA Advisory Board (DAB)

11. DAB Standards issued in 1998-1999 Quality Assurance Standards (QAS) 1. SCOPE 2. DEFINITIONS 3. QUALITY ASSURANCE PROGRAM 4. ORGANIZATION AND MANAGEMENT 5. PERSONNEL 6. FACILITIES 7. EVIDENCE (SAMPLE) CONTROL 8. VALIDATION 9. ANALYTICAL PROCEDURES 10. EQUIPMENT CALIBRATION AND MAINTENANCE 11. REPORTS 12. REVIEW 13. PROFICIENCY TESTING 14. CORRECTIVE ACTION 15. AUDITS 16. SAFETY 17. SUBCONTRACTOR OF ANALYTICAL TESTING FOR WHICH VALIDATED PROCEDURES EXIST http://www.fbi.gov/hq/lab/fsc/backissu/july2000/codis2a.htm http://www.fbi.gov/hq/lab/fsc/backissu/july2000/codis1a.htm Revised Quality Assurance Standards go into effect July 1, 2009

12. Checks and Controls on DNA Results CommunityFBI DNA Advisory Board’s Quality Assurance Standards (also interlaboratory studies) LaboratoryASCLD/LAB Accreditation and Audits AnalystProficiency Tests & Continuing Education Method/Instrument Validation of Performance (along with traceable standard sample) ProtocolStandard Operating Procedure is followed Data SetsAllelic ladders, positive and negative amplification controls, and reagent blanks are used Individual SampleInternal size standard present in every sample Interpretation of Result Second review by qualified analyst/supervisor Court Presentation of Evidence Defense attorneys and experts with power of discovery requests

13. Three Possible Outcomes Match – Peaks between the compared STR profiles have the same genotypes and no unexplainable differences exist between the samples. Statistical evaluation of the significance of the match is usually reported with the match report (see Chapter 21). Exclusion (Non-match) – The genotype comparison shows profile differences that can only be explained by the two samples originating from different sources. Inconclusive – The data does not support a conclusion as to whether the profiles match. This finding might be reported if two analysts remain in disagreement after review and discussion of the data and it is felt that insufficient information exists to support any conclusion. Butler, J.M. (2005) Forensic DNA Typing, 2 nd Edition, p. 385

14. What It Means to “Validate” An Analysis Process How reproducible is an analysis? –If the same evidence is examined multiple times, would the same conclusion be reached each time? How robust is an analysis? –Are results obtained every time (or a high percentage of the time) evidence is tested? How reliable is an analysis? –If known samples are examined, are the expected results obtained? –Are situations with insufficient evidence reaching conclusions of “no results”?

15. http://www.fbi.gov/hq/lab/fsc/ current/backissu.htm Forensic Science Publications

16. Chapter 13 – Points for Discussion How is validation helpful to the overall quality assurance program in a laboratory? What role do organizations like SWGDAM and ENFSI play in quality assurance? Why are reference materials valuable in aiding quality assurance?