1. Use of Microbial Risk Assessment in Decision-Making David Vose Consultancy 24400 Les Lèches Dordogne France www.risk-modelling.com Email David Vose's secretary David Vose Slide show on: www.risk-modelling.com/firstmicrobial.htm Note the 2 ‘l’s !

2. Slide 2 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making Introduction Applying CODEX guidelines in reality Difficulties Other ways of thinking Experience with microbial modelling Some survey results The Dutch experience Some US experience Modelling challenges Comparison of some complete models Reviewing a model in context

3. Slide 3 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making Microbial risk assessment is a scientifically-based process consisting of four steps: 1.Hazard Identification The identification of known or potential health affects associated with a particular agent; 2.Exposure Assessment The qualitative and/or quantitative evaluation of the degree of intake likely to occur; 3.Hazard Characterization The quantitative and/or qualitative evaluation of the nature of the adverse effects associated with biological, chemical and physical agents that may be present in food… For biological agents… a dose-response assessment should be performed if the data is available; 4.Risk Characterization Integration of Hazard Identification, Hazard Characterization and Exposure Assessment into an estimation of the adverse effects likely to occur in a given population, including attendant uncertainties. Codex Alimentarius Commission FAO/WHO (1995)

4. Slide 4 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making OIE experience OIE produced guidelines for animal import risk assessments (for the management of disease spread) Now in its second edition Guidelines were offered as a way to help member (including developing) countries understand how to perform a r.a. First Ed. guidelines were used too literally, both by analysts and lawyers, and found to be often impractical or irrelevant to the risk question Lesson: keep guidelines non-specific, encourage understanding rather than prescribing a formulaic approach Popular interpretation of CODEX guidelines suffer similarly

5. Slide 5 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making (1) Risk analysis uses observations about what we know to make predictions about what we don’t know. Risk analysis is a fundamentally science-based process that strives to reflect the realities of Nature in order to provide useful information for decisions about managing risks. Risk analysis seeks to inform, not to dictate, the complex and difficult choices among possible measures to mitigate risks... Society for Risk Analysis Principles for Risk Analysis

6. Slide 6 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making (2) Risk analysis seeks to integrate knowledge about the fundamental physical, biological, social, cultural, and economic processes that determine human, environmental, and technological responses to a diverse set of circumstances. Because decisions about risks are usually needed when knowledge is incomplete, risk analysts rely on informed judgment and on models reflecting plausible interpretations of the realities of Nature. We do this with a commitment to assess and disclose the basis of our judgments and the uncertainties in our knowledge. Society for Risk Analysis Principles for Risk Analysis

7. Slide 7 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making Current modelling Microbial QRA is a developing science We’re making a lot of progress, but it is still in infancy Mostly producing ‘farm-to-fork’ Models the whole system but very poorly Not designed to model any decision question well Often relies on poor data, surrogates, and guesses Almost never is a decision question posed beforehand Assessors have probably over-sold QRA’s usefulness Managers have expected too much

8. Slide 8 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making F2F Achilles’ Heels Very little data available, system being modelled is hugely complex! Uncertainty, variability, inter-individual variablility Take too long to complete, too easy to make mistakes F2F considers only pathogen on the food source E.g. not E.coli produced during life of animal, appearing in water, vegetables, farmers’ exposure Predictive microbiology still unreliable Broth data doesn’t translate well to food (usually overestimate, but some data – Tamplin, USDA – shows lag period can be shorter, e.g. E.coli in ground beef, Listeria in processed hams) Models often not based on physical/biological ideas, so we don’t learn Attenuation may not be death, and ignores reactivation of bacteria D-R models inadequate Don’t describe variability observed P(ill|dose, infected) = P(ill|infected)? Feeding trial data don’t match epi data – can hugely underestimate the risk Little cost-benefit analysis effort made Including actions affecting several risk issues Requires enormous resources – impractical for many countries

9. Slide 9 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making The lessons learnt from risk analysis experiences: 1.Risk management has not always been an integral part of risk analysis so far; 2.Risk managers should be trained to understand risk assessment, and risk assessors should be trained to explain their work; 3.Available data are often of limited use for risk assessment and communication of data needs between risk assessors, food scientists and risk managers is a critical issue; 4.The risk manager questions usually require rapid results, whereas (farm-to-fork) risk assessment projects require several years to complete. Solving this conflict requires open communication; 5.Uncertainty is often large. Dutch observations on past QRA Havelaar, Jansen (2002)

10. Slide 10 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making Our survey Internet based, voluntary participation, 39 valid responses

11. Slide 11 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making

12. Slide 12 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making

13. Slide 13 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making

14. Slide 14 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making Completion times of some farm-to-fork QRAs Final report Draft report Being revised Draft report Final report

15. Slide 15 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making Salmonella dose-response Epi and feeding trial comparison Review by Amir Fazil in FAO/WHO (2001) D-R mathematical models review by Haas (2002)

16. Slide 16 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making “Although the goal was to make the model comprehensive, it has some important limitations. It is a static model and does not incorporate possible changes in SE over time as either host, environment or agent factor change. For many variables, data were limited or nonexistent. Some obvious sources of contamination, such as food handlers, restaurant environment, or other possible sites of contamination on or in the egg (such as the yolk), were not included. And, as complex as the model is, it still represents a simplistic view of the entire farm-to-table continuum. Finally, the model does not yet separate our uncertainty from the inherent variability of the system. Much more work is needed to address this, and all other, limitations.” USDA-FSIS-FDA Salmonella Enteritidis

17. Slide 17 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making USDA-FSIS-FDA Salmonella Enteritidis Original model impetus was to evaluate effect of refrigeration temp from laying to retail on food safety Empirically must have little affect since it only deals with a few days in the life of an egg No cost-benefit attached Now being redone to focus on level of performance required for shell, and liquid egg pasteurisation i.e. much more decision focused

18. Slide 18 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making FDA Listeria risk assessment No specific decision questions attached Attempted to look at relative importance of a large list of Listeria-carrying foods Given the data available, perhaps the only method possible to estimate which food types contribute the greatest risk So a good QRA application

19. Slide 19 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making Remedies: focusing on decisions Consider what is known about the risk problem, and data available immediately or within acceptable time frame Use epi data as much as possible Collect more epi data (e.g. Japan, Denmark) Consider what analysis could be done with this knowledge i.e. a risk-based reasoned argument for evaluating particular actions Estimate the possible magnitude of benefit for a risk action Note that it may not be possible to evaluate all actions Perform a cost-benefit analysis on these actions Perform a Value of Information analysis Determines whether it is worth collecting more data before making a decision Consider strategy to validate whether predicted improvement occurs Train data producers to supply maximally useful data E.g. microbiologists taken more than one cfu from a plate More inter-agency unity E.g. Farm (APHIS)  Slaughter (FSIS)  Retail (FDA)

20. Slide 20 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making Make it as simple as possible: example Risk: Human illness from SE in eggs A shell-egg selection system proposed that will reduce by 30% the number of contaminated eggs going to market Currently, 30,000 people a year suffer from SE from eggs What will be the reduction in cases if the new system is implemented? Reduction in cases = 30%*30,000 = 9,000 people/year No need for models of D-R, bacterial growth, handling, etc. Vulnerability to assumptions smaller than from using F2F model

21. Slide 21 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making Campylobacter Risk Management and Assessment Dutch proposal The main objectives of the project are to advise on the effectiveness and efficiency of measures aimed at reducing campylobacteriosis in the Dutch population. The two key questions are: 1.What are the most important routes (quantifiable?)? 2.Which (sets of) measures can be taken to reduce the exposure to Campylobacter, what is their expected efficiency and societal support? An example of the way forward Havelaar, Jansen 2002

22. Slide 22 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making The target of the assessment is not limited to estimating the possible reduction in disease incidence but to evaluate both costs and benefits of possible interventions and to access their acceptance by stakeholders. Interventions with low social support will require more effort to uphold, which increases their costs and reduces their efficacy. The way forward – cont.

23. Slide 23 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making Danish Vet Service Salmonella QRA “ A Bayesian Approach to Quantify the Contribution of Animal-food Sources to Human Salmonellosis” - H ald, Vose, Koupeev (2002) Estimated number of cases of human salmonellosis in Denmark in 1999 according to source Model ranks food sources by risk. Easily updateable with each year’s data. Bayesian update improves estimate and checks validity of assumptions.

24. Slide 24 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making Fluoroquinolone-resistant Campylobacter risk assessment Model: Contaminated carcasses after slaughter plant * probability = affected people

25. Slide 25 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making Broiler house Transport Slaughter house Hanging Scalding Defeathering Evisceration Washing Chilling Export Chicken parts Whole chickens Chilled Frozen Import Catering Cross contamination Heat treatment Retail Consumer Cross contamination Heat treatment Dose response Further Processing Risk estimation Slaughterhouse model Consumer model Example of Farm-to-Fork model Campylobacter in poultry Draft report 2001 Institute of Food Safety and Toxicology Division of Microbiological Safety Danish Veterinay and Food Administration Behaves the same way as CVM model if prevalence is reduced

26. Slide 26 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making AHI model Fluoroquinolone resistant Campylobacter in poultry AHI / Cox 2000 A dynamic simulation model (i.e. follow the path of a random chicken) Used same data as CVM where available. Strong potential because it reduces model complexity (at the expense of simulation time)and easy to follow. Difficulty is availability of data to make model parameters meaningful. Most model parameters in current version represent nothing physical (‘factors’), so don’t enlighten us as to what actions to take. D-R and consumer handling difficulties remain.

27. Slide 27 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making Reviewing a risk assessment Risk assessment should be decision focused It is not appropriate to review a risk assessment independently from the question(s) the assessment is addressing Eg because a point is moot if the decision is insensitive to the argument It uses science but is not itself scientific research So we have to go with the best we’ve got

28. Slide 28 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making Finally - risk assessors should gain hands-on experience to ensure their models reflect the real world

29. Slide 29 David Vose Consultancy Ltd www.risk-modelling.com Microbial risk analysis in decision making References Haas, C.N., (2002), Conditional Dose-Response Relationships for Micro- organisms: Development and Applications. Risk Analysis 22 (3): 455-464. Havelaar, H. and J. Jansen, (2002), Practical Experience in the Netherlands with quantitative microbiological risk assessment and its use in food safety policy. Draft paper, RIVM, Bilthoven, The Netherlands. Hope, B.K., et al., (2002), An overview of the Salmonella Enteritidis Risk Assessment for Shell Eggs and Egg Products. Risk Analysis 22 (3): 455-464. Joint FAO/WHO Expert Consultation on the Application of Risk Analysis to Food Standards Issues (Joint FAO/WHO, 1995). Joint FAO/WHO Expert Consultation on Risk Assessment of Microbiological Hazards in Foods: Risk characterization of Salmonella spp. in eggs and broiler chickens and Listeria monocytogenes in ready-to-eat foods. (2001), FAO headquarters, Rome. Teunis, P.F.M. and A.H.Havelaar, (2001), The Beta-Poisson Dose-Response Model Is Not a Single-Hit Model. Risk Analysis 20 (4): 513-520.