1. The Strengths and Limitations of Regulatory Peer Review Dr. Heather E. Douglas Phibbs Assistant Professor of Science and Ethics University of Puget Sound

2. Difficulties Of Regulatory Peer Review High stakes of regulatory decisions Complex nature of regulatory documents

3. Range of Reasonable Expectations Upper Bound: Guarantee Truth Level 5: Best Interpretation of Available Evidence Level 4: Document Distinguishes Contested from Accepted Science Level 3: Recommendations for Further Study Level 2: Acceptable Review of Relevant Science Level 1: Minimal Scientific Literacy/Adequacy Lower Bound: Cosmetic Approval/Disapproval

4. The Spectrum of Reasonable Expectations Level 1: Basic Scientific Accuracy Needs: Basic relevant expertise among reviewers needed. Reviewers need not meet. Level 2: Good Review of Science Level 3: Recommendations for Study Level 4: Disputed Areas Acknowledged Level 5: Best Interpretation of Science

5. The Spectrum of Reasonable Expectations Level 1: Basic Scientific Accuracy Level 2: Good Review of Science Needs: Each relevant sub-field needs to be represented. Having experts meet is helpful, but not necessary. Level 3: Recommendations for Study Level 4: Disputed Areas Acknowledged Level 5: Best Interpretation of Science

6. The Spectrum of Reasonable Expectations Level 1: Basic Scientific Accuracy Level 2: Good Review of Science Level 3: Recommendations for Study Needs: Same composition as Level 2, but more time and effort is needed to develop recommendations. Panel meetings would be helpful here. Level 4: Disputed Areas Acknowledged Level 5: Best Interpretation of Science The Spectrum of Reasonable Expectations

7. Level 1: Basic Scientific Accuracy Level 2: Good Review of Science Level 3: Recommendations for Study Level 4: Disputed Areas Acknowledged Needs: At least a few experts from each sub-field are needed to delineate contested areas. Experts must meet together and discuss their views. Level 5: Best Interpretation of Science The Spectrum of Reasonable Expectations

8. Level 1: Basic Scientific Accuracy Level 2: Good Review of Science Level 3: Recommendations for Study Level 4: Disputed Areas Acknowledged Level 5: Best Interpretation of Science Needs: Experts must represent not only an appropriate range of expertise but an appropriate range of values. Panels must be balanced for both. Experts must meet to weigh values in the context of these judgments. The Spectrum of Reasonable Expectations

9. General Conclusions Increased expectations mean: - increased complexity of process - more care in selection of reviewers - reviewers will need more time - consensus less likely

10. Examples of Tensions in Regulatory Peer Review 1)Defining conflict of interest 2)Chasing consensus 3)Limited resources and peer review

11. Lessons: Regulatory peer review can at least help ensure there are no errors in the document. As one expects more than this, difficulties rise. As expectations rise, so should the resources and care committed to the review process.