What Is Peer Review? Dr. Heather E. Douglas Philip M. Phibbs Assistant Professor of Science and Ethics University of Puget Sound

Peer review is… a formal review by peers whose purpose is 1)To help improve the final product. 1)To perform a gate keeping function.

What is not peer review? “Non-peer” review Peer “non-(formal) review”

“Non-peer” review i.e., review by the general public Examples:public comment periods public reception of intellectual work Key difference:reviewers need not meet a standard of expertise

Peer “non-(formal) review” i.e., peer consultation Examples:sharing papers with colleagues discussions among colleagues Key differences:no formal review process peers consulted no longer independent

So, what is peer review? Different purposes Different processes Described by: Product under review Goal Standards Process Stakes

Dissertation Defense Product: Goal: Process: Stakes: Dissertation Produce competent peers Reviewers not blind Not really independent Do you get your degree? Does an incompetent PhD get out?

Product: Goal: Process: Stakes: Tenure file Keep competent peers Weed people out of institution Reviewers can be both independent and previous consultants Job security Permanent colleague Tenure evaluation

Key Differences with Product Peer Review 1) No opportunity to improve product post-review 2) Independence of reviewers doubtful Thus these review processes are very emotionally charged

Publication Review Product: Goal: Standard: Process: Stakes: Paper Publish interesting new work Guard limited publication resources Is this piece competent? Will it make a contribution to the field? Usually double blind 2-3 reviewers Maintain value of publication Make new work available

Product: Goal: Standard: Process: Stakes: Grant proposal Protect limited funds Support worthwhile work Is this project worth funding? Single blind Multiple reviewers Good use of funds Funding valuable work Grant review

Product: Goal: Standard: Process: Stakes : Regulatory document Ensure solid scientific basis for policy Is this the most accurate account of science? (Are the interpretations correct?) No blinding possible Usually many reviewers, often meeting as a group Are we making the right decision? Harm/help those affected Regulatory review

As the stakes get higher, the complexity of the review increases and thus the difficulty of the review process increases. Regulatory peer review has the highest stakes. That is the challenge of regulatory peer review. As the stakes get higher…

For more information see: Peerless Science; Peer Review and U.S. Science Policy by Daryl Chubin and Edward J. Hackett (1990). The Fifth Branch: Science Advisers as Policymakers by Shiela Jasanoff (1991). For more information see: