Validation of Microbiological Methods for Use in the Food Industry Brazilian Association for Food Protection 6 th International Symposium Sao Paulo, Brazil June 15 th, 2007

Introduction Hundreds of new methods developed each year Hundreds of new methods developed each year Pathogenic organisms Pathogenic organisms Non-Pathogenic organisms Non-Pathogenic organisms Detection Detection Identification Identification How do you know if you need a new method? How do you know if you need a new method? How do you decide if it is the right method for your purpose? How do you decide if it is the right method for your purpose?

Introduction Goal of methods evaluation is to find an innovative technology that will allow for quick and efficient detection and/or quanitation of pathogens and spoilage organisms Goal of methods evaluation is to find an innovative technology that will allow for quick and efficient detection and/or quanitation of pathogens and spoilage organisms

Performance Criteria The Three S’s The Three S’s Sensitivity Sensitivity What is the sensitivity of current method What is the sensitivity of current method What degree of sensitivity is needed What degree of sensitivity is needed Specificity Specificity What is the false positive rate What is the false positive rate What is the false negative rate What is the false negative rate Speed Speed What is speed of current method (samples processed/day) What is speed of current method (samples processed/day) How quickly are results needed How quickly are results needed

Performance Criteria Costs Costs What is cost of current method What is cost of current method What is cost of instrumentation What is cost of instrumentation What is cost of disposables/reagents What is cost of disposables/reagents What is the cost per test What is the cost per test Reagents Reagents Prep time Prep time Stability Stability Availability Availability Consistency (Quality Control) Consistency (Quality Control)

Performance Criteria Versatility Versatility Product only Product only Variety of food matrixes Variety of food matrixes Environmental samples only Environmental samples only Pathogens only Pathogens only Microorganisms only Microorganisms only Bacteria and/or Fungi Bacteria and/or Fungi Acceptability of method by scientific community and/or Regulators Acceptability of method by scientific community and/or Regulators AOAC, AOAC-RI, USDA-FSIS, FDA, AFNOR AOAC, AOAC-RI, USDA-FSIS, FDA, AFNOR

Performance Criteria Vendor company reputation Vendor company reputation First product on market First product on market Training Training Vendor provided training on site Vendor provided training on site How much, how long How much, how long Technical Service Technical Service Speed of service Speed of service Availability of service (24-7) Availability of service (24-7) Service contract required Service contract required

Technical Evaluation Objective Objective Justification (benefit of method to company) Justification (benefit of method to company) Acceptance Criteria Acceptance Criteria Material and Methods Material and Methods Test Media/Conditions Test Media/Conditions Microorganisms Microorganisms Genus, species, source Genus, species, source Inoculum preparation Inoculum preparation Inoculation Procedure Inoculation Procedure Statistical Analysis Statistical Analysis Results Results Next Steps Next Steps

Case Study #1 Dichloran-Rose Bengal Agar Yeast and Mold Method Evaluation

Dichloran-Rose Bengal Agar Yeast and Mold Method Evaluation Objective: Determine validity of a 2 day yeast and mold method using DRB agar incubated at 30C or 35C Objective: Determine validity of a 2 day yeast and mold method using DRB agar incubated at 30C or 35C Justification: Reduced product holding time, resulting in significant cost savings to the plant Justification: Reduced product holding time, resulting in significant cost savings to the plant Acceptance Criteria: Recovery efficiencies must be equivalent to the current 5 day PDA method Acceptance Criteria: Recovery efficiencies must be equivalent to the current 5 day PDA method

Dichloran-Rose Bengal Agar Yeast and Mold Method Evaluation Microorganisms: Microorganisms: Mold Cultures Mold Cultures A.niger, Penicillium spp., and Paecilomyces spp. A.niger, Penicillium spp., and Paecilomyces spp. Yeast Cultures Yeast Cultures Z.ballii, S.cerevisiae, and a plant isolate Z.ballii, S.cerevisiae, and a plant isolate Inoculum Preparation: Inoculum Preparation: Organisms were harvested from aPDA plates by washing with sterile water Organisms were harvested from aPDA plates by washing with sterile water 1ml from each individual mold or yeast suspension was added to 20 mls DI water 1ml from each individual mold or yeast suspension was added to 20 mls DI water Molds serially diluted Molds serially diluted Yeast adjusted to a spec reading of 1.00, then serially diluted Yeast adjusted to a spec reading of 1.00, then serially diluted

Dichloran-Rose Bengal Agar Yeast and Mold Method Evaluation Material and Methods: Material and Methods: product was inoculated with 100 cfu/g of target organisms product was inoculated with 100 cfu/g of target organisms 0.1ml of inoculated product surface plated onto each media (aPDA, DRBA) 0.1ml of inoculated product surface plated onto each media (aPDA, DRBA) aPDA incubated at 25C aPDA incubated at 25C Counted at 3 and 5 days Counted at 3 and 5 days DRBA incubated at 30C and 35C DRBA incubated at 30C and 35C Counted at 2, 3, 4, and 5 days Counted at 2, 3, 4, and 5 days

Dichloran-Rose Bengal Agar Yeast and Mold Method Evaluation Statistical Analysis: An analysis of variance (AOV) was done to test if the total counts for DRB at 2 and 5 days was significantly different from aPDA at 5 days Statistical Analysis: An analysis of variance (AOV) was done to test if the total counts for DRB at 2 and 5 days was significantly different from aPDA at 5 days

Dichloran-Rose Bengal Agar Yeast and Mold Method Evaluation Results: Results: DRB at 2 days-30C was statistically equivalent to aPDA at 5 days for mold recovery DRB at 2 days-30C was statistically equivalent to aPDA at 5 days for mold recovery Molds were pale in color; Penicillium spp. was white on DRB (green on aPDA). The other 2 test molds were pale yellow Molds were pale in color; Penicillium spp. was white on DRB (green on aPDA). The other 2 test molds were pale yellow Yeast counts on DRB at 30C were significantly lower than counts on aPDA at 2 and 5 days Yeast counts on DRB at 30C were significantly lower than counts on aPDA at 2 and 5 days Mold and Yeast counts were significantly lower on DRB at 35C vs. aPDA Mold and Yeast counts were significantly lower on DRB at 35C vs. aPDA

Dichloran-Rose Bengal Agar Yeast and Mold Method Evaluation Conclusion: Conclusion: Due to overall decreased recovery of yeast and mold, and the mold visual observations; the Dichloran-Rose Bengal Agar Yeast and Mold recovery medium is not recommended. Due to overall decreased recovery of yeast and mold, and the mold visual observations; the Dichloran-Rose Bengal Agar Yeast and Mold recovery medium is not recommended.

Case Study #2 Rapid Check Salmonella Test Kit Evaluation

Rapid Check Salmonella Test Kit Evaluation Objective: Determine validity of the Strategic Diagnostics Inc. Rapid Check antibody lateral flow method for the detection of Salmonella in comparison to the BAX PCR test method Objective: Determine validity of the Strategic Diagnostics Inc. Rapid Check antibody lateral flow method for the detection of Salmonella in comparison to the BAX PCR test method Justification: Reduce testing cost, false positives rate and technician time Justification: Reduce testing cost, false positives rate and technician time

Rapid Check Salmonella Test Kit Evaluation Acceptance Criteria: Acceptance Criteria: Speed; shorter time to results vs. PCR? Speed; shorter time to results vs. PCR? Sensitivity; greater or equivalent to PCR? Sensitivity; greater or equivalent to PCR? Specificity; greater or equivalent to PCR Specificity; greater or equivalent to PCR Cost Cost Less than or equal to BAX PCR system Less than or equal to BAX PCR system Cost per test Cost per test Versatility; food products only, environmental samples only, or both? Versatility; food products only, environmental samples only, or both?

Rapid Check Salmonella Test Kit Evaluation Organisms and Inoculum Preparation: Organisms and Inoculum Preparation: A cocktail of 5 Salmonella spp. A cocktail of 5 Salmonella spp. A cocktail of 7 non-Salmonella spp. A cocktail of 7 non-Salmonella spp. E.coli (2), Citrobacter, Bacillus, Klebsiella, Enterobacter (2) E.coli (2), Citrobacter, Bacillus, Klebsiella, Enterobacter (2) Individual cultures grown overnight in BHI at 35C Individual cultures grown overnight in BHI at 35C Salmonella strains pooled, diluted to 100cfu/ml Salmonella strains pooled, diluted to 100cfu/ml Non-Salmonella strains pooled, diluted to 1,000 cfu/ml Non-Salmonella strains pooled, diluted to 1,000 cfu/ml

Rapid Check Salmonella Test Kit Evaluation Methods: Methods: Inoculation of samples Inoculation of samples With Salmonella With Salmonella With non-Salmonella strains With non-Salmonella strains With both With both Pre-enrichment of samples Pre-enrichment of samples Traditional medium; Lactose for 24 hours Traditional medium; Lactose for 24 hours SDI medium for 5 hours SDI medium for 5 hours Secondary enrichment Secondary enrichment Tetrathionate for 24 hours Tetrathionate for 24 hours

Rapid Check Salmonella Test Kit Evaluation Methods (cont): Methods (cont): BAX PCR analysis BAX PCR analysis 3 hour re-growth 3 hour re-growth Cell lysis Cell lysis 4-8 hour PCR cycle 4-8 hour PCR cycle SDI lateral flow assay SDI lateral flow assay Load 150ul onto SDI cartridge Load 150ul onto SDI cartridge Develop for 10 minutes Develop for 10 minutes

Rapid Check Salmonella Test Kit Evaluation Results: Results: Sensitivity Sensitivity Results were more consistent with SDI when recovering at the threshold level (1000 cfu/ml in the TT broth) Results were more consistent with SDI when recovering at the threshold level (1000 cfu/ml in the TT broth) Equivalent results with both methods above the threshold level Equivalent results with both methods above the threshold level SDI 5 hour pre-incubation media did not consistently support growth above the threshold level (acceptance criteria) SDI 5 hour pre-incubation media did not consistently support growth above the threshold level (acceptance criteria) Specificity Specificity No cross reactivity with non-Salmonella organisms with either method No cross reactivity with non-Salmonella organisms with either method

Rapid Check Salmonella Test Kit Evaluation Results: Speed Results: Speed BAX-PCR SDI w/ 5 hour medium SDI Enrichment 24 hours 5 hours 24 hours Secondary 18 hours Re-growth 3 hours 0 hours Cell lysis 0.5 hours 0 hours Analysis time 8 hours 10 minutes Total 53.5 hours 23 hours, 10 minutes 42 hours, 10 minutes

Rapid Check Salmonella Test Kit Evaluation Conclusions: Conclusions: SDI shown to be as sensitive as BAX-PCR SDI shown to be as sensitive as BAX-PCR 5 hour medium not recommended 5 hour medium not recommended No cross reactivity observed with SDI No cross reactivity observed with SDI SDI gave results sooner than PCR SDI gave results sooner than PCR PCR has more steps, more prone to technician error PCR has more steps, more prone to technician error Some degree of subjectivity with SDI Some degree of subjectivity with SDI SDI easier to use; 1 step inoculation of 1 single cartridge SDI easier to use; 1 step inoculation of 1 single cartridge

Rapid Check Salmonella Test Kit Evaluation Conclusions: Conclusions: SDI can be successfully used for food and environmental samples SDI can be successfully used for food and environmental samples No additional equipment needed (heat blocks, thermal cycler) No additional equipment needed (heat blocks, thermal cycler) Cost per test of SDI less than BAX-PCR Cost per test of SDI less than BAX-PCR SDI approved for use in place of PCR SDI approved for use in place of PCR Appropriate for use by labs analyzing a smaller number of samples Appropriate for use by labs analyzing a smaller number of samples

Value of Method Validation Need to validate method on your intended product; rule out matrix interference Need to validate method on your intended product; rule out matrix interference Determine minimum regulatory requirements (AOAC, AFNOR, etc) Determine minimum regulatory requirements (AOAC, AFNOR, etc) Determine what is the right method for your lab based on volume of testing and number of technicians Determine what is the right method for your lab based on volume of testing and number of technicians Base selection of methodology on need Base selection of methodology on need Sensitivity Sensitivity Specificity Specificity Speed Speed Cost Cost Lab space Lab space