1. Method Validation and Verification: An Overview Patricia Hanson, Biological Administrator I Florida Department of Agriculture and Consumer Services, Food Safety Microbiology Laboratory

2. Introduction 17 years in the microbiology section of the Florida Department of Agriculture and Consumer Services, Food Laboratory Laboratory moved from primarily cultural methods in 1998 to a leader in the implementation of new technology in 2015 Numerous partnerships with state, federal and industrial customers to validate and implement new methods

3. Goals Distinguish between method verification and method validation Based on our laboratory’s experience, identify the critical components of both a method verification and a method validation Offer insight to approaches we have found useful, specifically in the microbiology section Share lessons learned

4. Verification vs Validation Verification is a demonstration that your laboratory can perform a Standard Method or other well documented method and produce acceptable results Validation is both a demonstration that your lab can run the method and that the method or change to a method is fit for purpose

5. Aspects of a Verification Run entire method, start to finish, exactly as it is written Include both positive and negative samples Represent all matrix types Be spiked at a level that is fit for the purpose of the method if spiked samples are used

6. Verifications can be ongoing…. For the purpose of this discussion, “Verification” is being used to describe what the laboratory does prior to implementing a method in the laboratory The term “Verification” is often also used to describe the ongoing process of verifying a method with each run – in our lab this it the use of Process Controls, Cultural Controls, and Media Controls

7. Validation Laboratory developed method Matrix Extensions ANY technically significant change to a Standard Method or previously validated method

8. Overview of an Existing Guideline Level One: Urgent Use, Single Lab Validation Level Two: Single Lab Validation Level Three: Multi-Lab Validation Level Four: Full Collaborative Study

9. Level 2: Single Lab Validation Highlights 50 different strains of target organism, if available 30 different strains of “nearest neighbor” organisms At least two target analyte levels 20 replicates per food type per analyte level Perform side by side if possible

10. Aspect of a Verification Be extensive enough to show that the method is fit for the intended use and meets the customer’s needs

11. What is “Extensive Enough” Items to be considered:  Matrix: number and types  Target: source and levels  Method Evaluation: side by side or use of standard reference material When considering these items during the development of the validation, keep in mind what is needed to show the method is fit for the intended use and meets the customer needs.

12. Matrices Validation or verification needs to be performed on each food type Meat (raw, smoked, cooked, etc) Vegetable (whole, cut, sprout, etc) Dairy (liquid, solid, etc) Other (spices, candies, etc)

13. Replicates Exact number depends on method Verification, normally triplicate Validation, normally six replicates per level Important to do the same number of replicates of “un-spiked”

14. Naturally Contaminated Samples Naturally contaminated pros and cons Best test of the method Level of analyte is unknown Little or no control of analyte level

15. Spiked Samples Spiked sample pros and cons Ability to control organism level Methods often perform differently on spiked samples than on naturally contaminated samples

16. Aging of Spiked Cultures Spiking samples 48-72 hours prior to initiation of testing can better represent naturally occurring contamination Sometimes the cells are so stressed, they die off Consider the method or part of method being validated and type of validation

17. Selection of Organisms Select at least one organism for each target “Nearest Neighbor” organisms – not often used but good to consider

18. “Side by Side” Validations Normally involves running the method being validated side by side with a Standard or established method Twice the work but very valuable for determining if the “new” method is equivalent or better than the Standard or established method Best choice when changing part of a method – for example a new enrichment broth but same screening method

19. How Much Confirmation? Screening Method Validations: Confirm everything You can get by with an abbreviated confirmation, for example just biochemical panel and/or serology but do enough to show you “got out what you put in.”

20. Why Do So Many Confirmations? Ensure you can confirm samples that screen positive Ensure samples you can confirm screen positive Spiked samples that don’t screen positive and don’t confirm point to limitations of the method or problems with the spiking procedure

21. Data Evaluation The measures used to evaluate data depend on the purpose and type of the method and the needs of the customer

22. Qualitative Data Evaluation Detection Limits – can the method detect the target at the levels required by the customer? Specificity – can the method detect the target in the matrix? Selectivity – can the method differentiate between the target and other analytes in the matrix? Robustness – can the method be repeated on the same sample with the same results?

23. Quantitative Data Evaluation Detection Limits, Specificity, Selectivity, and Robustness Precision – Does the method produce the same results as established method? Linearity – Does the method produce proportional results with in the customer’s target range?

24. Personnel Considerations Personnel designing and approving verifications and validations should be knowledgeable about the technology as well as the needs of the lab

25. Personnel Considerations Personnel carrying out verification and validations should be competent in all techniques Work on verification and validation studies can be used to demonstrate competency but should not be used as training

26. Pit-falls of Method Validation Repeating tests until you get the result you want Drifting from method validation into method development Adjusting criteria for acceptance AFTER the data is obtained

27. Overcoming Pit-falls Have a well thought out, documented and approved plan and stick to it Go back and start over, incorporating any changes into new validation All data should still be included in evaluation

28. Documentation Approval of plan prior to running Raw Data Interpretation of raw data Final approval from customer Reference validation in SOP If SOP is a modified Standard Method, list deviations from the Standard Method in a section in the SOP

29. References U. S. Department of Health and Human Services, Food and Drug Administration (2014, August 29). Volume II – Methods, Method Verification and Validation ORA- LAB.5.4.5. Retrieved from http://www.fda.gov/ScienceResearch/FieldScience/ucm1 71877.htm http://www.fda.gov/ScienceResearch/FieldScience/ucm1 71877.htm U. S. Department of Health and Human Services, Food and Drug Administration (2011, September 8). Guidelines for the Validation of Analytical Methods for the Detection of Microbial Pathogens in Foods. Retrieved from http://www.fda.gov/downloads/ScienceResearch/FieldSc ience/UCM273418.pdf http://www.fda.gov/downloads/ScienceResearch/FieldSc ience/UCM273418.pdf

30. Questions