1. FSIS’s Needs for Peer Review of Risk Assessments Carol Maczka, Ph.D. Senior Scientist for Risk Assessment FSIS/USDA September 30, 2003 Presented at JIFSAN/SRA/RAC Symposium On Peer Review of Risk Assessment

2. FSIS Peer Review Process to date Lessons Learned Needs and Challenges

3. Peer Review Mechanisms Salmonella Enteriditis Risk Assessment in Shell Eggs and Egg Products (1995) Inter- and Intra-Agency review by specialists Independent review by select subject area specialists Publications in peer review literature E. coli 0157:H7 Risk Assessment in Ground Beef Inter- and Intra-Agency review by specialists NACMCF review Independent review by select subject area specialists Public comment External peer review by National Academies of Science

4. Peer Review Mechanisms (Continued) Risk of E. coli 0157:H7 in Tenderized versus Non-Tenderized Steaks/Roasts Inter-Agency review NACMCF review Harvard BSE Risk Assessment Commission risk assessment (Independence) Inter- and Intra-Agency review External peer review using independent contractor Listeria Risk Assessment - Product Contamination from Food Contact Surfaces Inter- and Intra-Agency review ORACBA review Public comment External peer review using independent contractor ORACBA/OMB review

5. Peer Review Mechanisms (Continued) Other Risk Assessments (Currently in Progress) Perfringens Risk Assessment in RTE Meat and Poultry Products Risk Assessment for Salmonella Enteriditis in Shell Eggs and Salmonella spp in Egg Products Salmonella Risk Assessment in Raw Beef and Poultry Products Salmonella, E. coli 0157:H7 Risk Assessment in RTE Meat and Poultry Products External Peer Review using Independent Contractor

6. FSIS Peer Review Process to Date Intra-AgencyReviewORACBA IndependentExternal Peer Review Inter-AgencyReviewRA PublicComment ORACBA OMB R RRR R = Revised in response to comments

7. Elements of a “Good” Peer Review Process Independent External Peer Review especially for major regulations or influential information Access to broad range of scientific expertise (modelers, subject area experts) Timely and cost-effective review Appropriate staging of peer review and public comment to avoid unnecessary revisions Reviewers are given a focused charge and are invited to provide general comments. At a minimum, reviewers should be asked to comment on: Overarching logical structure of the model; Validity and appropriateness of the data used, reasonableness of assumptions made, model’s mathematics and equations, whether risks have been appropriately characterized; key sources of variability and uncertainty identified; critical assumptions; data gaps. Computerized models should be audited. Reviewers are instructed to avoid policy determinations Lessons Learned

8. Elements of a “Good” Peer Review Process Reviewers are provided with relevant background information on potential sources of controversy Reviewers are held accountable Review is balanced Reviewers do not have a conflict of interest Procedures for documenting response to comments; comment/response document Outside firm (entity) supervises the review. Comments are provided to Agency blinded SOPs for conducting peer reviews Lessons Learned (Continued)

9. Needs and Challenges Large pool of external peer reviewers (“expert registry”) Requirements for Peer Reviewers Scientific and Technical Expertise/Experience Multiple areas of expertise (e.g., modelers, epidemiologists, microbiologists) No real or perceived conflict of interests No advocated positions (balance) Have not conducted multiple peer reviews for the Agency in recent years How can an “expert registry” be created ? How and When are the pool of potential candidates screened for bias/conflict of interests

10. Needs and Challenges (Continued) Appropriate Mechanisms for Peer Review Contract with Independent Body National Academies of Science Outside Firm NACMCF Risk Assessment Consortium Consortium of Universities ? Pro Bono Others ? Need to ensure Timely/Cost Effective/Independent Peer Review Mechanism Standard Operating Procedures for Conducting Peer Review Need to ensure consistency with other agencies conducting significant regulatory review

11. Summary of Major Needs Pool of External Reviewers from which to select Mechanisms for accessing peer reviewers Establish SOPs for external peer review