1. www.metcardio.org SURVIVING THE PEER REVIEW PROCESS Giuseppe Biondi Zoccai Division of Cardiology, University of Turin, Turin, Italy Meta-analysis and Evidence-based medicine Training in Cardiology (METCARDIO), Ospedaletti, Italy

2. www.metcardio.org LEARNING GOALS What is peer review, and why have we to survive it through? What should you do and not do when actually peer reviewing other colleagues’ works? What should you do and not do when surviving through peer review? Case studies

3. www.metcardio.org WHY AM I GIVING YOU THIS LECTURE? Peer reviews I have completed since 2003 expected

4. www.metcardio.org LEARNING GOALS What is peer review, and why have we to survive it through? What should you do and not do when actually peer reviewing other colleagues’ works? What should you do and not do when surviving through peer review? Case studies

5. www.metcardio.org DEFINITION Peer vb intr. 1. to look intently with or as if with difficulty. 2. to appear partially or dimly. n. 1. a person who is an equal in social standing, rank, age, etc. Collins Dictionary of the English Language, London & Glasgow: Collins, 1979

6. www.metcardio.org ORIGINS OF PEER REVIEW Peer review was born in the 17th century in the UK, likely thanks to Henry Oldenburg, the founder of Philosophical Abstractions (1665) He originally introduced the practice of soliciting opionions on manuscripts from more knowledgeable external colleagues

7. www.metcardio.org NOTABLE EXCEPTIONS Not all manuscripts undergo thorough peer review. Notable examples? – Albert Einstein’s “Annus Mirabilis” papers, published in 1905 in Annalen der Physik by Max Planck (father of quantum theory and Nobel prize winner), who read the papers and decided altogether to publish them – more humbly… Biondi-Zoccai et al, Int J Cardiol 2005;100:119-23

8. www.metcardio.org WHAT’S WRONG WITH PEER REVIEW? Unreliable Unfair Fails to truly validate or authenticate Unstandardized Idiosyncratic Open to every sort of bias

9. www.metcardio.org WHAT’S EVEN WORSE? Stifles innovation Perpetuates the status quo Rewards the prominent but punishes the weak Unnecessarily delays dissemination Very expensive Insufficiently tested

10. www.metcardio.org YES, BUT… It has been said that democracy is the worst form of government except all those other forms that have been tried from time to time – W. Churchill The same applies to peer review

11. www.metcardio.org WHAT ARE ITS PROS? Filters bad from good, and better from less good, as well as possibly identifying misleading and false research Protects patients Guides authors to improve the quality of their article, and improve their whole research approch Authenticates work, assuring quality Improves readability Broadens participation and dialogue

12. www.metcardio.org PARAPHRASING GROUCHO I would never enter into a club that would accept me as a member… Groucho Marx, 1980-1977

13. www.metcardio.org LEARNING GOALS What is peer review, and why have we to survive it through? What should you do and not do when actually peer reviewing other colleagues’ works? What should you do and not do when surviving through peer review? Case studies

14. www.metcardio.org TYPICAL REVIEW FLOW

15. www.metcardio.org ARE PEER REVIEWERS UNBIASED? Mahoney et al, CTR 1977

16. www.metcardio.org WILL PEER REVIEW LET YOU SLEEP? McNutt et al, JAMA 1990 On average a review will take you around 3 hours (actually I usually complete one in 45’)

17. www.metcardio.org SHOULD YOU BE BLINDFOLDED? Godlee et al, JAMA 1998

18. www.metcardio.org WHO ARE THE BEST REVIEWERS? Kliever et al, AJR 2005

19. www.metcardio.org WHO ARE THE BEST REVIEWERS? Kliever et al, AJR 2005 Other (also weak) predictors: Coming from good institutions Known to the editors Had methodological training (statistics & epidemiology)

20. www.metcardio.org WHO ARE THE WORST REVIEWERS? Opthof et al, Cardiovasc Res 2002 Italian peer reviewers don’t like Italian manuscripts!

21. www.metcardio.org THREE PIVOTAL QUESTIONS FOR PEER REVIEW Do I understand it? Are the question and the methods clearly explained? Do I believe it? Are the conclusions justified by the data and are the methods valid? Do I care? Is the question important and interesting? Wager et al, How to survive peer review. BMJ Books 2002

22. www.metcardio.org CHECKLIST FOR PEER REVIEWERS: ISSUES FOR COMMENT Importance of research question Originality of work Delineation of strengths and weaknesses of methodology/experimental/statistical approach/interpretation of results Writing style and figure/table presentation Ethical concerns (animal/human) Benos et al, Advan Physiol Educ 2003

23. www.metcardio.org SUBTLETIES OF PEER REVIEW: PRIORITY – i.e. how to kill a paper Priority means novelty, originality, and likelihood of generating interest, irrespective from quality, validity, and methodology Manuscript can be judged as low, mid, high, or top priority Some journals (e.g. the Journal of Cardiovascular Medicine) use scores, e.g. ranging from 0 to 100, with 100 meaning top priority

24. www.metcardio.org Decision means recommending a specific editorial handling of the manuscript, and can be distinguished in: – accept as is – accept/reconsider after minor revisions – accept/reconsider after major revisions – reject but reconsider on a de novo basis – reject SUBTLETIES OF PEER REVIEW: DECISION

25. www.metcardio.org SUBTLETIES OF PEER REVIEW: DECISION Accept as is: it can be published as it stands Accept after minor revisions: it can be published EVEN IF not all my comments are taken into account Accept/reconsider after major revisions: it can be published ONLY IF all my comments are taken into account Reject but reconsider on a de novo basis: it must be changed altogether, and priority also reappraised after resubmission Reject: just send it back, it ain’t worth it

26. www.metcardio.org SUBTLETIES OF PEER REVIEW: DECISION Accept as is: it can be published as it stands Accept after minor revisions: it can be published EVEN IF not all my comments are taken into account Accept/reconsider after major revisions: it can be published ONLY IF all my comments are taken into account Reject but reconsider on a de novo basis: it must be changed altogether, and priority also reappraised after resubmission Reject: just send it back, it ain’t worth it

27. www.metcardio.org TYPICAL EDITOR’S EVALUATION OF PEER REVIEW AND PEER REVIEWER Thoroughness and comprehensiveness Timeliness Citing appropriate evidence to support comments made to author Providing constructive criticism Objectivity Clear statement to editor as to the appropriateness and priority of research for publication Benos et al, Advan Physiol Educ 2003

28. www.metcardio.org LEARNING GOALS What is peer review, and why have we to survive it through? What should you do and not do when actually peer reviewing other colleagues’ works? What should you do and not do when surviving through peer review? Case studies

29. www.metcardio.org DON’T PANIC

30. www.metcardio.org SHOULD YOU TRUST PEER REVIEWERS? Opthof et al, Cardiovasc Res 2002

31. www.metcardio.org IDENTIFYING THE MOST APPROPRIATE TARGET Wager et al, How to survive peer review. BMJ Books 2002

32. www.metcardio.org IDENTIFYING THE MOST APPROPRIATE TARGET Wager et al, How to survive peer review. BMJ Books 2002

33. www.metcardio.org COVER LETTER

34. www.metcardio.org AFTER SUBMITTING PAPER Store all data and documentation surrounding data analysis in durable and appropriately referenced form Store the original data—questionnaires, data collection sheets, CD’s Keep as long as readers may reasonably expect to ask questions where you need to reference this data

35. www.metcardio.org AFTER SUBMITTING PAPER Store 5-10 years--normal All references to where data held and how archived logged in handbook Include all details so study can be repeated Names, locations of electronic data files Data bases, data recodes, data analysis

36. www.metcardio.org AFTER SUBMITTING PAPER Once paper submitted now property of journal Editor has total discretion over who has access Peer editors sometimes pass on papers to colleagues for review Confidentiality not always maintained

37. www.metcardio.org REPLYING TO REVIEWERS COMMENTS Do not ignore comments, but make most if not all of changes Be calm, objective, and polite even if reviewer’s comments were harsh Deconstruct each of the messages into individual items Respond to each item thoughtfully Make responses clear

38. www.metcardio.org REPLYING TO REVIEWERS COMMENTS Don’t have to fully accept suggestions but must give reasons that will convince editor your opinion is reasonable Be pragmatic and not dismissive of reviewer’s work Explain just enough to enable you to survive Benefit from it, and learn also how to become a competent reviewer

39. www.metcardio.org REBUTTAL LETTER

40. www.metcardio.org AFTER RESUBMISSION OF REVISION No guarantee will be published Editor will consider new version and your replies to comments Editorial process can be subjective, and sometimes downgrade priority -> rejection Reviewer’s comments only one factor Editor may reject paper even if reviewer’s comments were minor Editor has absolute discretion

41. www.metcardio.org IF REJECTED CAN APPEAL If you think reviewer’s overlooked or misunderstood something important Appeal by writing a letter stating your case –> rebuttal/complaint letter Rare decision overturned but it does happen If appealing—send new copy of paper— rejected papers do not remain on file (i.e. de novo submission)

42. www.metcardio.org WHAT TO DO AFTER REJECTION Reviewers critical of basic methods—may need to rethink study and do further data analysis Reviewers critical on style and presentation— fix problems before resubmitting to another journal Three repeat rejections—completely reassess entire approach or search for appropriate target

43. www.metcardio.org WHEN ACCEPTED Receive page proofs—typeset copy of work— how looks in journal May take several months to receive Time for final check Journal usually send proof reading instructions you must follow Usually standard proofreading marks

44. www.metcardio.org TYPICAL REASONS FOR FINAL REJECTION 1.Fundamentally weak hypothesis 2.Lack of clinical relevance 3.Old knowledge with no new or useful material 4.Two or three of reasons 1–3 5.Reasonable text, but images are of very poor quality, are inappropriate, or are incorrectly interpreted 6.Too many methodologic errors 7.Hypothesis adequate, but poor study design, methodology, or statistics 8.Lacking in logic; initial premise not logically supported by methods and results Ehara et al, AJR 2007

45. www.metcardio.org TYPICAL REASONS FOR FINAL REJECTION (CONT.) 9.None of the other reasons, but reviewers do not like the article 10.Previously published 11.Sample population too small or biased to justify results and conclusion 12.Well written but better suited for another journal 13.Major language problems; English not primary language of author 14.Too poorly written, phrased, or presented 15.Failure to follow journal guidelines 16.Lack of correlation between purpose and results 17.Poor statistics, beyond salvage Ehara et al, AJR 2007

46. www.metcardio.org TEN TIPS TO FOR SURVIVAL 1.Properly organize the manuscript 2.Clearly state the study question and rationale 3.Explain methods in a systematic fashion 4.Structure methods and results in a similar manner 5.Make the discussion session concise 6.Explain if – and why – your study results are important 7.Avoid overinterpreting 8.Explain the limitations 9.Account for unexpected findings 10.Fully incorporate reviewers’ suggestions Provenzale et al, AJR 2007

47. www.metcardio.org LEARNING GOALS What is peer review, and why have we to survive it through? What should you do and not do when actually peer reviewing other colleagues’ works? What should you do and not do when surviving through peer review? Case studies

48. www.metcardio.org THE LOVERBOY REVIEWER Lotrionte et al, Am J Cardiol 2008

49. www.metcardio.org THE LOVERBOY REVIEWER Lotrionte et al, Am J Cardiol 2008

50. www.metcardio.org THE INQUIRING REVIEWER Biondi-Zoccai et al, Am Heart J 2008

51. www.metcardio.org THE INQUIRING REVIEWER Biondi-Zoccai et al, Am Heart J 2008

52. www.metcardio.org THE DELUSIONAL REVIEWER Sheiban et al, J Am Coll Cardiol 2009

53. www.metcardio.org THE DELUSIONAL REVIEWER Sheiban et al, J Am Coll Cardiol 2009

54. www.metcardio.org THE NASTY REVIEWER Sheiban et al, J Interven Cardiol 2008

55. www.metcardio.org THE NASTY REVIEWER Sheiban et al, J Interven Cardiol 2008

56. www.metcardio.org THE PESSIMISTIC REVIEWER Biondi-Zoccai et al, Am Heart J 2008

57. www.metcardio.org THE PESSIMISTIC REVIEWER Biondi-Zoccai et al, Am Heart J 2008

58. www.metcardio.org THE ASK-TOO-MUCH REVIEWER 27 COMMENTS JUST FROM THE STATISTICAL REVIEWER!!! HOW CAN YOU TACKLE THIS? Biondi-Zoccai et al, BMJ 2006

59. www.metcardio.org THE ASK-TOO-MUCH REVIEWER 27 COMMENTS JUST FROM THE STATISTICAL REVIEWER!!! HOW CAN YOU TACKLE THIS? WITH A LOT OF PATIENCE!!! Biondi-Zoccai et al, BMJ 2006

60. www.metcardio.org “There is no way to get experience except through experience.”

61. www.metcardio.org TAKE HOME MESSAGES What a very bad reviewer once told Amir Lerman (Mayo Clinic, USA): Delete the first phrase of the second section of the Discussion (“It remains to be speculated that…”. Better, print it, put it in a frame and hang it in front of your desk to remind you never to repeat such mistake….

62. www.metcardio.org TAKE HOME MESSAGES Active or passive peer review remains a time- consuming and painful process, but still the best available Active peer review is best summarized by Matthew 19, 16-19: Thou shalt love thy neighbor as thyself Passive peer review should be based on patience, perserverance, and sound methods

63. www.metcardio.org Thank you for your attention For any correspondence: gbiondizoccai@gmail.com For these and further slides on these topics feel free to visit the metcardio.org website: http://www.metcardio.org/slides.html gbiondizoccai@gmail.com http://www.metcardio.org/slides.html