1. 1 MEASURING THE EFFECTIVENESS OF THE NATION’S FOODSERVICE AND RETAIL FOOD PROTECTION SYSTEM

2. 2 PRESENTATION OBJECTIVES Purpose of FDA’s Study Review What the Study IS and IS NOT Review the Design of the Study Preview of 2003 Results Present Recommendations Purpose of FDA’s Study Review What the Study IS and IS NOT Review the Design of the Study Preview of 2003 Results Present Recommendations

3. 3 BACKGROUND FDA National Retail Food Steering Committee Government Performance and Results Act (1993) “performance plans … measurable indicators” FDA National Retail Food Steering Committee Government Performance and Results Act (1993) “performance plans … measurable indicators”

4. 4 INTRODUCTION AND PURPOSE Agency effort to change behaviors / practices related to foodborne illness Identify food safety practices and employee behavior most in need of attention Agency effort to change behaviors / practices related to foodborne illness Identify food safety practices and employee behavior most in need of attention

5. 5 INTRODUCTION AND PURPOSE Establish a national baseline on the occurrence of foodborne illness risk factors Effort to collect sufficient data to measure patterns in the occurrence of foodborne illness risk factors over time Establish a national baseline on the occurrence of foodborne illness risk factors Effort to collect sufficient data to measure patterns in the occurrence of foodborne illness risk factors over time

6. 6 STUDY TIME LINE 1998 1 ST Data Collection Report of the FDA Retail Food Program Database of Foodborne Illness Risk Factors (2000) Established a Baseline for 9 different foodservice and retail food facility types 1998 1 ST Data Collection Report of the FDA Retail Food Program Database of Foodborne Illness Risk Factors (2000) Established a Baseline for 9 different foodservice and retail food facility types

7. 7 STUDY TIME LINE 2003 2 nd Data Collection FDA Report on the Occurrence of Foodborne Illness Risk Factors in Selected Institutional Foodservice, Restaurant and Retail Food Store Facility Types (2004) Begins the process for establishing multiple data collection periods 2003 2 nd Data Collection FDA Report on the Occurrence of Foodborne Illness Risk Factors in Selected Institutional Foodservice, Restaurant and Retail Food Store Facility Types (2004) Begins the process for establishing multiple data collection periods

8. 8 STUDY TIME LINE 2008 3 rd Data Collection will be conducted Report prepared for 2009 will begin the process of analyzing the three data sets (1998, 2003, and 2008) to assess improvement or regression changes from the 1998 Baseline

9. 9 WHAT THE REPORT IS AND IS NOT IS – a national assessment of the relative strengths & weaknesses of food safety systems designed to control the occurrence of foodborne illness risk factors IS NOT – a correlation of the occurrence of foodborne illness risk factors with the actual incidence of human illness IS – a national assessment of the relative strengths & weaknesses of food safety systems designed to control the occurrence of foodborne illness risk factors IS NOT – a correlation of the occurrence of foodborne illness risk factors with the actual incidence of human illness

10. 10 WHAT THE REPORT IS AND IS NOT IS– an assessment that used the 1997 Food Code provisions as the standard upon which observations of food safety practices are based IS NOT– a determination of the industry’s OR a specific establishment’s regulatory compliance with prevailing laws & regulations IS– an assessment that used the 1997 Food Code provisions as the standard upon which observations of food safety practices are based IS NOT– a determination of the industry’s OR a specific establishment’s regulatory compliance with prevailing laws & regulations

11. 11 METHODOLOGY Scope: 3 Industry segments / 9 facility types: 1.Institutions Hospitals Nursing Homes Elementary Schools Scope: 3 Industry segments / 9 facility types: 1.Institutions Hospitals Nursing Homes Elementary Schools

12. 12 METHODOLOGY Scope: 3 Industry segments / 9 facility types: 2.Restaurants Fast Food Full-Service Scope: 3 Industry segments / 9 facility types: 2.Restaurants Fast Food Full-Service

13. 13 METHODOLOGY Scope: 3 Industry segments / 9 facility types: 3. Retail Food Stores -- 4 departments: Deli Meat and Poultry Seafood Produce Scope: 3 Industry segments / 9 facility types: 3. Retail Food Stores -- 4 departments: Deli Meat and Poultry Seafood Produce

14. 14 METHODOLOGY 2003 Data Collection: – 926 Inspections – 15,516 Observations – Standardized Specialists conducted the inspections 2003 Data Collection: – 926 Inspections – 15,516 Observations – Standardized Specialists conducted the inspections

15. 15 METHODOLOGY –Selection of Project Locations –Selection of Establishments (Comparison Lists) –Selection of Project Locations –Selection of Establishments (Comparison Lists)

16. 16 METHODOLOGY –Confidentiality of Selected Establishments –Observational vs. Regulatory inspections –Confidentiality of Selected Establishments –Observational vs. Regulatory inspections

17. 17 BACKGROUND CONTRIBUTING FACTORS Food from Unsafe Source Inadequate Cooking Improper Holding/Time-Temp. Contaminated Equipment/Protection from Contamination Poor Personal Hygiene Other “Chemical Hazards” Food from Unsafe Source Inadequate Cooking Improper Holding/Time-Temp. Contaminated Equipment/Protection from Contamination Poor Personal Hygiene Other “Chemical Hazards”

18. 18

19. 19 METHODOLOGY Data Collection – Records status for all individual data items IN – In Compliance OUT – Out of Compliance N.O. – Not Observed N.A. – Not Applicable Data Collection – Records status for all individual data items IN – In Compliance OUT – Out of Compliance N.O. – Not Observed N.A. – Not Applicable

20. 20 DATA ANALYSIS Three levels of data analysis are conducted for each of the 9 facility types A. Out of Compliance percentage for each individual data item B. Out of Compliance percentage for each foodborne illness risk factor C. Overall IN Compliance percentage for all 42 data items Three levels of data analysis are conducted for each of the 9 facility types A. Out of Compliance percentage for each individual data item B. Out of Compliance percentage for each foodborne illness risk factor C. Overall IN Compliance percentage for all 42 data items

21. 21 Determining the Out of Compliance percentage for each DATA ITEM Percent Out of Compliance = Total Out of Compliance Observations for a Data Item Total Out of Compliance Observations for a Data Item Total number of Observations (IN and OUT) for the Data Item Total number of Observations (IN and OUT) for the Data Item X 100%

22. 22 Determining Out of Compliance percentages for each DATA ITEM The proportion of establishments where the data item was found Out of Compliance at least once when the practice or procedure could be observed The proportion of establishments where the data item was found Out of Compliance at least once when the practice or procedure could be observed

23. 23 Determining Out of Compliance Percentages FOODBORNE ILLNESS RISK FACTORS Observations of DATA ITEMS FOODBORNE ILLNESS RISK FACTOR Observations of DATA ITEMS FOODBORNE ILLNESS RISK FACTOR

24. 24

25. 25 Determining the Out of Compliance percentage for each FOODBORNE ILLNESS RISK FACTOR Percent Out of Compliance = Total Out of Compliance Observations for a Risk Factor Total Out of Compliance Observations for a Risk Factor Total number of Observations (IN and OUT) for the Risk Factor Total number of Observations (IN and OUT) for the Risk Factor X 100%

26. 26 Example Out of Compliance Percentage Poor Personal Hygiene - Risk Factor Poor Personal Hygiene 5 DATA ITEMS – Data collection Form 12A 13A 14A 15A 15B Poor Personal Hygiene - RISK FACTOR Poor Personal Hygiene 5 DATA ITEMS – Data collection Form 12A 13A 14A 15A 15B Poor Personal Hygiene - RISK FACTOR

27. 27 Determining Overall IN Compliance Percentages - FACILITY TYPES - 6 RISK FACTOR CATEGORIES Overall In Compliance % Facility Types 6 RISK FACTOR CATEGORIES Overall In Compliance % Facility Types

28. 28

29. 29 FDA Report on the Occurrence of Foodborne Illness Risk Factors in Selected Institutional Foodservice, Restaurant and Retail Food Store Facility Types (2004) FDA Report on the Occurrence of Foodborne Illness Risk Factors in Selected Institutional Foodservice, Restaurant and Retail Food Store Facility Types (2004)

30. 30 2004 Report Report should be read and the data interpreted as a separate stand-alone assessment The report provides insights into food safety practice and employee behaviors needing priority attention Report should be read and the data interpreted as a separate stand-alone assessment The report provides insights into food safety practice and employee behaviors needing priority attention

31. 31 Foodborne Illness Risk Factor in Need of Priority Attention (2000 Report) Improper Holding/Time and Temperature Poor Personal Hygiene Contaminated Equipment / Protection from Contamination Improper Holding/Time and Temperature Poor Personal Hygiene Contaminated Equipment / Protection from Contamination

32. 32 2004 Report Data Results and Discussion For each of the 9 facility types the Data Results will present: A.Overall Out of Compliance percentage for each of the foodborne illness risk factors B. Individual data Items that comprise each of the risk factors that are in need of priority attention For each of the 9 facility types the Data Results will present: A.Overall Out of Compliance percentage for each of the foodborne illness risk factors B. Individual data Items that comprise each of the risk factors that are in need of priority attention

33. 33 2004 Report Data Results and Discussion For each of the 9 facility types the Data Results will present: C. A national strengths/weaknesses assessment of management systems for controlling the occurrence of foodborne illness risk factors For each of the 9 facility types the Data Results will present: C. A national strengths/weaknesses assessment of management systems for controlling the occurrence of foodborne illness risk factors

34. 34

35. 35

36. 36

37. 37

38. 38

39. 39 Project Limitations Field Limitations -- Time of the Inspection -- Length of the Inspection Field Limitations -- Time of the Inspection -- Length of the Inspection

40. 40 Project Limitations Statistical Limitations Some comparisons not supported by statistical Design -- Region vs. Region, State vs. State Statistical Limitations Some comparisons not supported by statistical Design -- Region vs. Region, State vs. State

41. 41 Project Limitations Statistical Limitations Some comparisons not supported by statistical Design -- Subcategories of Facility Types (e.g., chains of restaurants, retail food stores) Statistical Limitations Some comparisons not supported by statistical Design -- Subcategories of Facility Types (e.g., chains of restaurants, retail food stores)

42. 42 NEW AREAS OF STUDY - Supplemental Data Items - Study is designed to track improvements/regression in food safety practices and employee behaviors Supplement items designed to track impact of changes in the Food Code on compliance percentages in the study Study is designed to track improvements/regression in food safety practices and employee behaviors Supplement items designed to track impact of changes in the Food Code on compliance percentages in the study

43. 43

44. 44 NEW AREA OF STUDY Impact of a Certified Food Protection Manager 4 facility types had Overall IN Compliance percentages that were significantly higher in establishments with certified managers Impact of a Certified Food Protection Manager 4 facility types had Overall IN Compliance percentages that were significantly higher in establishments with certified managers

45. 45 NEW AREA OF STUDY - Fast Food Restaurants - Full Service Restaurants - Meat & Poultry Departments - Produce Departments

46. 46

47. 47 NEW AREA OF STUDY Impact of a Certified Food Protection Manager There was NO risk factor for which the IN Compliance % for establishments without a certified manager exceeded the % for establishments with a certified manager in a statistically significant manner Impact of a Certified Food Protection Manager There was NO risk factor for which the IN Compliance % for establishments without a certified manager exceeded the % for establishments with a certified manager in a statistically significant manner

48. 48 NEW AREA OF STUDY DATA COLLECTION TIME - determined for each facility type - only includes actual time in the facility - does not include travel or report writing

49. 49

50. 50 Pathways to Reach the Goals Risk Factors + Intervention Strategies + Performance Measures FDA Foodborne Illness Risk Factor Study Standardization & Certification Program Standards

51. 51 RECOMMENDATIONS FOR INDUSTRY ACTIVE MANAGERIAL CONTROL OF FOODBORNE ILLNESS RISK FACTORS ACTIVE MANAGERIAL CONTROL OF FOODBORNE ILLNESS RISK FACTORS

52. 52 ACTIVE MANAGERIAL CONTROL Purposeful incorporation of specific actions or procedures by industry management to attain control of foodborne illness risk factors Purposeful incorporation of specific actions or procedures by industry management to attain control of foodborne illness risk factors

53. 53 RECOMMENDATIONS FOR INDUSTRY Develop and implement Standard Operating Procedures (SOPs) to address FBI risk factors

54. 54 RECOMMENDATIONS FOR INDUSTRY Provide employees with specific training and equipment to implement the SOPs

55. 55 RECOMMENDATIONS FOR INDUSTRY Incorporate critical limits and measurable standards for control of FBI risk factors in SOPs

56. 56 RECOMMENDATIONS FOR INDUSTRY Establish monitoring procedures that focus on critical processes and practices

57. 57 RECOMMENDATIONS FOR INDUSTRY Identify methods to routinely assess the effectiveness of the SOPs

58. 58 RECOMMENDATIONS FOR INDUSTRY FDA guidance document for industry Managing Food Safety: A Guide for Voluntary Use of HACCP Principles for Operators of Foodservice and Retail Establishments www.cfsan.fda.gov/~dms/hret-toc.html FDA guidance document for industry Managing Food Safety: A Guide for Voluntary Use of HACCP Principles for Operators of Foodservice and Retail Establishments www.cfsan.fda.gov/~dms/hret-toc.html

59. 59 RECOMMENDATIONS FOR REGULATORY Use risk-based inspection methodology - risk factors should be the primary focus of every inspection Provide flexible work schedules Use risk-based inspection methodology - risk factors should be the primary focus of every inspection Provide flexible work schedules

60. 60 RECOMMENDATIONS FOR REGULATORY Properly train and equip field personnel Document compliance determination (IN; OUT; N.O.; N.A.) Properly train and equip field personnel Document compliance determination (IN; OUT; N.O.; N.A.)

61. 61 RECOMMENDATIONS FOR REGULATORY Establish a dialog with industry foodservice and retail food store operators Recognize existing industry Quality Assurance Systems and Training Programs Establish a dialog with industry foodservice and retail food store operators Recognize existing industry Quality Assurance Systems and Training Programs

62. 62 RECOMMENDATIONS FOR REGULATORY Take appropriate corrective action –Obtain immediate corrective action at the time of inspection for risk factors found out of compliance Take appropriate corrective action –Obtain immediate corrective action at the time of inspection for risk factors found out of compliance

63. 63 RECOMMENDATIONS FOR REGULATORY Take appropriate corrective action –Assist in developing Standard Operating Procedure and Risk Control Plans designed to attain long-term managerial control of risk factors Take appropriate corrective action –Assist in developing Standard Operating Procedure and Risk Control Plans designed to attain long-term managerial control of risk factors

64. 64 IMPLEMENT A CONSISTENT AND EFFECTIVE ENFORCEMENT PROTOCOL Develop enforcement procedures Ensure credibility Develop enforcement procedures Ensure credibility

65. 65 VOLUNTARY NATIONAL RETAIL FOOD REGULATORY PROGRAM STANDARDS Standards of Excellence for Continuous Improvement Standards of Excellence for Continuous Improvement

66. 66 Voluntary National Retail Food Regulatory Program Standards Regulatory Foundation 1. Regulatory Foundation 2. Staff Training 3. HACCP Principles-based Inspection Program 4. Inspection Uniformity 5. Foodborne Illness and Food Security Preparedness and Response Preparedness and Response Regulatory Foundation 1. Regulatory Foundation 2. Staff Training 3. HACCP Principles-based Inspection Program 4. Inspection Uniformity 5. Foodborne Illness and Food Security Preparedness and Response Preparedness and Response

67. 67 Voluntary National Retail Food Regulatory Program Standards 6. Compliance and Enforcement 7. Industry and Community Relations 8. Program Support and Resources 9. Self Assessment 6. Compliance and Enforcement 7. Industry and Community Relations 8. Program Support and Resources 9. Self Assessment

68. 68 Voluntary National Retail Food Regulatory Program Standards Identify program areas where an agency can have the greatest impact on retail food safety. Promote wider application of effective FBI risk factor intervention strategies Identify program areas where an agency can have the greatest impact on retail food safety. Promote wider application of effective FBI risk factor intervention strategies

69. 69 Voluntary National Retail Food Regulatory Program Standards Assist in identifying program areas most in need of additional resources

70. 70 Standards as a Tool for Continuous Improvement *Self Assess Improve Plan Measure

71. 71 FDA Retail Food Program Resource Disk Contains all FDA’s technical guidance for retail food protection programs: –National Retail Food Program Standards –FDA’s 2000 Baseline Report –Software Program & Guidance documents for jurisdictions to use to conduct their own foodborne illness risk factor studies Contains all FDA’s technical guidance for retail food protection programs: –National Retail Food Program Standards –FDA’s 2000 Baseline Report –Software Program & Guidance documents for jurisdictions to use to conduct their own foodborne illness risk factor studies

72. 72 FDA Retail Food Program Resource Disk –2001 Food Code (AND 2003 Food Code Supplement) –FDA Standardization Procedures –HACCP @ Retail Guides –Plan Review Guide –Temporary Events Guides –2001 Food Code (AND 2003 Food Code Supplement) –FDA Standardization Procedures –HACCP @ Retail Guides –Plan Review Guide –Temporary Events Guides

73. 73 PROGRAM GOAL ACTIVE MANAGERIAL CONTROL OF FOODBORNE ILLNESS RISK FACTORS ACTIVE MANAGERIAL CONTROL OF FOODBORNE ILLNESS RISK FACTORS