1. Lessons Learned from the 5 MFRPS Pilot States Oregon Wednesday, March 11, 2015 8:45 AM – 9:45 AM San Diego, CA Oregon Department of Agriculture Food Safety Program Gesinee Tolman MFRPS Technical Coordinator & HACCP Specialist

2. Stephanie Page Food Safety & Animal Health Program Director Frank Barcellos Food Program Manager Monica Durazo Field Operations Manager Terry Hill Field Operations Manager Gesinee Tolman Technical Coordinator & HACCP Specialist Jingyun Duan Compliance and Recall Coordinator Susan Kendrick Education Specialist ODA MFRPS Team Members John Burr Manufacturing Specialist Eric Edmunds Program Analyst Sarah Schwab IT Liaison

3. History Audits– Quick Recaps Lessons Learned Overview

4. Back in September 2010 at the WAFDO Educational Conference Ellen Buchanan

5. History  Enrolled with MFRPS as one of the pilot states in 2007  First audit by T. Ravelli: 2008  Consultation audit by E. Laymon: 2011  Consultation audit by A. Kohl: 2012  36-month PAVA by T. Ravelli: 2012  Awarded MFRPS Grant since August 2012  60-month PAVA by I. Bevill: 2014

6. ODA’s MFRPS Implementation Progress Standard 1: Regulatory Foundation Standard 1 Regulatory Foundation 2008 1 st Audit by T. Ravelli Team 2011 Consultatio n Audit by E. Laymon 2012 ODA’s Self- Assessment as Submitted for Contract Amendment 2011-12 FY 2012 Consultation Audit by A. Kohl 2012 36-Month PAVA by T. Ravelli Team 2014 60-Month PAVA by I. Bevill Team Self- Assessment Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Implementation Full  Partial ☐ Full  Partial ☐ Full  Partial ☐ Full  Partial ☐ Full  Partial ☐ Full ☐ Partial  * Conformance P * Oregon does not adopt FD&D Act by reference and has not performed a legal review to to determine if State laws are equivalent in effect to the current Federal laws. => ODA is working with Oregon Dept. of Justice to get the legal review done by the end of July, 2015.

7. * Partial--In 2012, 1) Not everyone had attended FDA150, FDA151, and FDA170 or completed all of the ORA U courses. This item was included on the strategic plan. 2) Basic food inspection field training was not conducted within 18 months of start date. 3) ODA did not have any current sign-offs on joint inspections for new staff based on staffing levels since 07-1-2011. Standard 2 Training Program 2008 1 st Audit by T. Ravelli Team 2011 Consultation Audit by E. Laymon 2012 ODA’s Self- Assessment as Submitted for Contract Amendment 2011-12 FY 2012 Consultation Audit by A. Kohl 2012 36-Month PAVA by T. Ravelli Team 2014 60-Month PAVA by I. Bevill Team Self- Assessment Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Implementation Full ☐ Partial  Full ☐ Partial  Full ☐ Partial  Full ☐ Partial  * Full  Partial ☐ Full  Partial ☐ Conformance Yes ODA’s MFRPS Implementation Progress Standard 2: Training Program

8. ODA’s MFRPS Implementation Progress Standard 3: Inspection Program Standard 3 Inspection Program 2008 1 st Audit by T. Ravelli Team 2011 Consultation Audit by E. Laymon 2012 ODA’s Self- Assessment as Submitted for Contract Amendment 2011-12 FY 2012 Consultation Audit by A. Kohl 2012 36-Month PAVA by T. Ravelli Team 2014 60-Month PAVA by I. Bevill Team Self- Assessment Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Implementation Full  Partial ☐ Full  Partial ☐ Full  Partial ☐ Full ☐ Partial  * Full ☐ Partial  ** Full  Partial ☐ Conformance P *** * 1) ODA did not have written policies & procedures for inspecting food plants (the 22 items). 2) ODA did not have written procedures for identifying and maintaining records about recall info. 3) ODA did not have a records retention policy/schedule for personnel records. ** Personnel were conducting juice HACCP inspections without advanced food inspection training. *** High risk firms are not always inspected at the frequency identified – due to limited resources

9. ODA’s MFRPS Implementation Progress Standard 4: Inspection Audit Program Standard 4 Inspection Audit Program 2008 1 st Audit by T. Ravelli Team 2011 Consultation Audit by E. Laymon 2012 ODA’s Self- Assessment as Submitted for Contract Amendment 2011-12 FY 2012 Consultation Audit by A. Kohl 2012 36-Month PAVA by T. Ravelli Team 2014 60-Month PAVA by I. Bevill Team Self- Assessment Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Implementation Full ☐ Partial  Full ☐ Partial  Full ☐ Partial  Full  Partial ☐ Full  Partial ☐ Full  Partial ☐ Conformance P * * Questions removed from Worksheet 4.4 and Appendix 4.7. Added “Not Applicable” to Appendix 4.5, 4.6 and 4.7. - All appendix and worksheets were corrected after the 60-month PAVA and are equivalent to FDA forms.

10. ODA’s MFRPS Implementation Progress Standard 5: Food-related Illness and Outbreaks, and Response Standard 5 Food-related Illness and Outbreaks, and Response 2008 1 st Audit by T. Ravelli Team 2011 Consultation Audit by E. Laymon 2012 ODA’s Self- Assessment as Submitted for Contract Amendment 2011-12 FY 2012 Consultation Audit by A. Kohl 2012 36-Month PAVA by T. Ravelli Team 2014 60-Month PAVA by I. Bevill Team Self- Assessment Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Implementation Full  Partial ☐ Full  Partial ☐ Full  Partial ☐ Full ☐ Partial  * Full ☐ Partial  * Full ☐ Partial  * Conformance P * ODA does not have a MOU that encompasses all elements of Standard 5. (5.3) - ODA is continuing to work with Oregon Health Authority to update and complete the MOU through future meetings.

11. ODA’s MFRPS Implementation Progress Standard 6: Compliance and Enforcement Standard 6 Compliance and Enforcement 2008 1 st Audit by T. Ravelli Team 2011 Consultation Audit by E. Laymon 2012 ODA’s Self- Assessment as Submitted for Contract Amendment 2011-12 FY 2012 Consultation Audit by A. Kohl 2012 36-Month PAVA by T. Ravelli Team 2014 60-Month PAVA by I. Bevill Team Self- Assessment Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Implementation Full ☐ Partial  Full ☐ Partial  Full ☐ Partial  Full ☐ Partial  * Full  Partial ☐ Full ☐ Partial  ** Conformance P * 1) ODA did not run a report on the critical and chronic violators. 2) ODA had some timelines established, but not for all enforcement actions. 3) ODA had not conducted a performance review of all enforcement actions. ** ODA is not meeting all timelines delineated in our policies for follow-up inspections for enforcement actions – due to limited resources

12. ODA’s MFRPS Implementation Progress Standard 7: Industry and Community Relations Standard 7 Industry and Community Relations 2008 1 st Audit by T. Ravelli Team 2011 Consultation Audit by E. Laymon 2012 ODA’s Self- Assessment as Submitted for Contract Amendment 2011-12 FY 2012 Consultation Audit by A. Kohl 2012 36-Month PAVA by T. Ravelli Team 2014 60-Month PAVA by I. Bevill Team Self- Assessment Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Implementation Full  Partial ☐ Full  Partial ☐ Full  Partial ☐ Full ☐ Partial  * Full  Partial ☐ Full  Partial ☐ Conformance Yes * ODA’s outreach form did not address the evaluation and critique sections of Appendix 7.

13. ODA’s MFRPS Implementation Progress Standard 8: Program Resources Standard 8 Program Resources 2008 1 st Audit by T. Ravelli Team 2011 Consultation Audit by E. Laymon 2012 ODA’s Self- Assessment as Submitted for Contract Amendment 2011-12 FY 2012 Consultation Audit by A. Kohl 2012 36-Month PAVA by T. Ravelli Team 2014 60-Month PAVA by I. Bevill Team Self-Assessment Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Implementation Full ☐ Partial  Full  Partial ☐ Full ☐ Partial  Full ☐ Partial  * Full  Partial ☐ Full  Partial ☐ Conformance P ** * 1) ODA had a calculation for the number of field staff for the whole program, but had not calculated the field staff for just the manufactured foods. 2) ODA did not include the MIG thermometer and pasteurization kits calibration procedures. This info was listed in the training plan, but the language should be strengthened. ** ODA does not have a procedure for calibration of thermometers – Corrected after the 60-month PAVA

14. ODA’s MFRPS Implementation Progress Standard 9: Program Assessment Standard 9 Program Assessment 2008 1 st Audit by T. Ravelli Team 2011 Consultation Audit by E. Laymon 2012 ODA’s Self- Assessment as Submitted for Contract Amendment 2011-12 FY 2012 Consultation Audit by A. Kohl 2012 36-Month PAVA by T. Ravelli Team 2014 60-Month PAVA by I. Bevill Team Self- Assessment Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Implementation Full ☐ Partial  Full ☐ Partial  Full  Partial ☐ Full  Partial ☐ Full ☐ Partial  * Full  Partial ☐ Conformance Yes * In the strategic plan, projected completion dates for each task were not identified.

15. ODA’s MFRPS Implementation Progress Standard 10: Laboratory Support Standard 10 Laboratory Support 2008 1 st Audit by T. Ravelli Team 2011 Consultation Audit by E. Laymon 2012 ODA’s Self- Assessment as Submitted for Contract Amendment 2011-12 FY 2012 Consultation Audit by A. Kohl 2012 36-Month PAVA by T. Ravelli Team 2014 60-Month PAVA by I. Bevill Team Self- Assessment Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Implementation Full ☐ Partial  Full ☐ Partial  Full ☐ Partial  Full ☐ Partial  * Full ☐ Partial  * Full ☐ Partial  * Conformance P * ODALS is not ISO 17025 accredited - ODALS is working towards the accreditation using FDA ISO CAP funds.

16. Current ODA’s Overall Self-Assessment- 2015 Standard 1Standard 2Standard 3Standard 4Standard 5 Self-Assessment Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Implementation Full ☐ Partial  Full  Partial ☐ Full  Partial ☐ Full  Partial ☐ Full ☐ Partial  Standard 6Standard 7Standard 8*Standard 9 Standard 10 Self-Assessment Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Complete  Incomplete ☐ Implementation Full ☐ Partial  Full  Partial ☐ Full ☐ Partial  Full  Partial ☐ Full ☐ Partial  * ODA’s Self-Assessment is different from the 60-month PAVA result (Full).

17. Lessons Learned Lessons learned is a pretty big category. ODA were doing many good things before the standards. ********************************************************** Here are the three (3) highlights; 1. Documenting what we do and, therefore; we have been able to identify gaps in our program. 2. Reviving a lot of items that were fading away over time in our program. 3. Creating and adopting new things that we didn’t have before.

18. Appendix 2.2-6: Staff Specialties for Inspections and Qualified Trainers 1. Documenting What We Do - We have been able to identify gaps in our programs i.e. “Training Program” --has allowed us to make modifications that in the end provide a more comprehensive program - Our inspectors are better trained than they were 10 years ago. This has also helped us towards our goal of consistency among inspectors.

19. Food Sampling SOPs 2. Reviving the Old and the Fades

20. - Review and revised the current P&P of all our enforcement actions. - Established a separate P&P of timeline for EAs Follow-up. 2. Reviving the Old and the Fades

21. Reasons for Recall: 2013 Pathogens (5): Hepatitis A, Cryptosporidium, L. monocytogenes Under-processed (9) Meat products without USDA inspection (1) Adulteration (2) Plastic inclusion (1) 2014 Pathogens (9) Campylobacter, Salmonella, L. monocytogenes Undeclared allergens (7) Product spoilage (2) Lower level of nutrients (1) 3. Creating and Adopting New Items

23. ** Outreach time was not tracked in 2009 and 2010. * Types of outreach activities have been tracked since 2009. 3. Creating and Adopting New Items

24. Lessons Learned The biggest negative is resources; All of the standards take a tremendous amount of time and staff hours to administer. Meeting time with FDA, audit time and reporting time have been required. Once the grant money runs out - although these tasks keep us accountable, they take a lot of time and staff hours.

25. 25 Stephanie P. Frank B. Monica D. Terry H. Jingyun D. Susan K. John B. Eric E.Sarah S. Gesinee T.

26. Contact Information Gesinee Tolman MFRPS Technical Coordinator & HACCP Specialist Oregon Department of Agriculture Food Safety Program Email: gtolman@oda.state.or.us