1. http//www.cytojournal.com 1 st Peer-Reviewer Retreat 2006 During USCAP Annual Meeting, Hyatt Regency Atlanta, Atlanta, Georgia February 12, 2006

2. Vinod B. Shidham, MD, FIAC, FRCPath Executive editor & coeditor-in-chief, CytoJournal (www.cytojournal.com) Director of Cytopathology Fellowship Training Program & FNAB Service Director of Fine Needle Aspiration Biopsy Service Medical College of Wisconsin (MCW) Milwaukee, WI 53226 vshidham@mcw.edu CytoJournal Peer-Review process: goals and resources

3. Goals ►Maintain high standards of peer-review process for CytoJournal ► Create a pool of best peer-reviewers for CytoJournal ► Appreciate the pivotal role of reviewers in publishing good science ► Emphasize significance of efficient quick turnover

4. Online submission of the manuscript to the editorial office Abstract directed to the executive editor by Managing editor with academic support / participation by trainee and faculty colleagues Monitoring/Academic editor is identified by the Executive editor (Managing editor will follow the progress) Abstract sent to Monitoring/Academic Editor At least 2 ad hoc reviewers are identified and communicated to Managing editor Manuscript is forwarded for Peer Review The comments by the reviewers sent to Editorial office are forwarded by Managing editor to be communicated to the Monitoring/Academic editor or the Executive editor Final decision (Revise*/Accept/Decline) is taken by both or one of the Editors-in- chief **, based on the reviewers comments and monitoring editor’s comments. If revision is indicated the manuscript is re-evaluated based on reviewer’s and / or Monitoring/Academic editor’s comments and final decision as to Accept/Decline by both or one of the Editors-in-chief ** is communicated to the author. Shidham VB, Cafaro A, Atkinson BF.Shidham VB, Cafaro A, Atkinson BF. CytoJournal joins 'open access' philosophy. Cytojournal. 2004 Jul 29;1(1):1. Free full text available at: http://www.cytojournal.com/content/1/1/1http://www.cytojournal.com/content/1/1/1

5. Limitations ►Although fundamental to scientific literature, PR has been largely an amateur process (1) ►Only a few biomedical experts are formally trained in critical analysis of manuscript (2) 1. BMJ training for peer-reviewers. BMJ 2004;328:658. 2. Callaham M, Schiger D, Copper RJ. An instructional guide for peer-review of Biomedical manuscripts. http://www3.us.elsevierhealth.com/extractor/graphics/em- acep/index.html

6. History ►In place only since 1950’s ►Evolved independently at individual journals ►Traditionally an ‘amateur’ volunteers without training ►Efficacy not proven ►Science as we know is product of PR 1. Schroter S, Black N, Evans S, Carpenter J, Godlee F, Smith R. Effects of training on quality of peer review: randomised controlled trial. BMJ 2004;328:673-5. 2. Davidoff F. Improving peer review: who’s responsible? BMJ 2004;328:657-8.

7. Role of PR in modern science ►Credibility to research publication ►Builds careers ►Facilitates grants, promotions ►Ultimately shapes clinical care of patients ►Catalyst for free scientific communication ►Quality assurance mechanism- identify flaws before publication ►Editorial tool- filter out ‘bad’ excess information ►‘Validation’ of science (in eyes of many readers)

8. Efficacy of PR ►Hyopthesis- PR is able to steer the quality of research ►Scientific scrutiny- little research ►Hard to demonstrate objectively- if it improves quality of published science ►Most believe- PR introduces consistency and quality. ►Compare- high quality journals with self published quack information ►Like democracy- It is not very good, but better than alternatives.

9. Weaknesses of PR process ►Part time enterprise of volunteers ►Lack of formal training or knowledge about resources ►Wide variation in qualifications and motivations ►PRs evaluating manuscript are few (1 to 3)- with low inter-rater agreement ►Individual biases of reviewers (pro or against) ►Bias favors positive than negative results ►Quality of reviewers is not assessed and tracked ►Can NOT detect fraud.

10. Product of PR process ►Detection of errors, omissions, misstatements. ►Monitor ethics in research ►Improve quality of data reporting ►Enhance readability of article ►Improve accuracy of claims and conclusions ►But, statistical errors are not usually detected by PR

11. Peer Reviewers Selection- How? ►Lack of standardized approach ►Usually based on word of mouth recommendations from EB and other experts in the field ►PubMed search ►Expertise in science does not equate with reviewer skills ►Retreats such as this may introduce objectivity ►Motivation and commitment for steering best science in the field is pivotal

12. Peer Reviewers- How are they trained? ►Almost never trained ►Graduate training in epidemiology and study design is believed to improve PR quality. ►Most journals DO NOT screen or train reviewers ►Impact of PR Training workshops is not proven ►It is recommended, at least to orient PRs and evaluate them on regular basis. ►Regular participation at such retreats appears to be favorable to achieve PR of high standards.

13. CytoJournal Peer Reviewers Motivated and committed for high standards of peer-review process Efficient with quick but high quality peer review for rapid turn around. ► ►

14. Resources for Peer-Review guidance ►On-line resources: 1. On-line training: Callaham M, Schiger D, Copper RJ. An instructional guide for peer-review of Biomedical manuscripts. http://www3.us.elsevierhealth.com/extractor/graphics/em-acep/index.html 2. Training package for BMJ peer reviewers: http://bmj.bmjjournals.com/advice/peer_review/ ►Reading material: 1. Schroter S, Black N, Evans S, Carpenter J, Godlee F, Smith R. Effects of training on quality of peer review: randomised controlled trial. BMJ 2004;328:673-5. 2. Davidoff F. Improving peer review: who’s responsible? BMJ 2004;328:657-8. 3. Purcell G, Donovan SL, Davidoff F. Measuring the quality of editorial peer review. JAMA 2002;287:2786-2790. 4. Peer Review issues of JAMA in March 1990, July 1994, July 1998, July 2002.

15. Resources for Peer-Review guidance ►On-line resources: 1. On-line training: Callaham M, Schiger D, Copper RJ. An instructional guide for peer-review of Biomedical manuscripts. http://www3.us.elsevierhealth.com/extractor/graphics/em-acep/index.html 2. Training package for BMJ peer reviewers: http://bmj.bmjjournals.com/advice/peer_review/ ►Reading material: 1. Schroter S, Black N, Evans S, Carpenter J, Godlee F, Smith R. Effects of training on quality of peer review: randomised controlled trial. BMJ 2004;328:673-5. 2. Davidoff F. Improving peer review: who’s responsible? BMJ 2004;328:657-8. 3. Purcell G, Donovan SL, Davidoff F. Measuring the quality of editorial peer review. JAMA 2002;287:2786-2790. 4. Peer Review issues of JAMA in March 1990, July 1994, July 1998, July 2002. ►On-line resources: 1. On-line training: Callaham M, Schiger D, Copper RJ. An instructional guide for peer-review of Biomedical manuscripts. http://www3.us.elsevierhealth.com/extractor/graphics/em-acep/index.html

20. PubMed indexed, peer-reviewed cytopathology journal Available free on-line at- (www.CytoJournal.com)