Cleveland State University ESC 720 Writing in Electrical and Computer Engineering Peer Review Dan Simon 1

Peer Review Outline 1.Overview 2.How to conduct a peer review 3.Is the peer review process unethical? 4.Does the peer review process enforce orthodoxy? 2

Peer Review What is peer review? Journal or conference editor receives a submitted paper Editor performs initial check for quality Editor sends paper to a few experts for review Editor receives reviews and makes a decision – Accept – Reject – Modify (does not apply to conferences due to tight timeline) 3

Peer Review Why should you be a peer reviewer? Moral obligation for those who submit papers Opportunity to see cutting-edge research Opportunity for networking 4

Peer Review Are you an appropriate reviewer? Objectively evaluate your familiarity with the subject 5

Peer Review Outline 1.Overview 2.How to conduct a peer review 3.Is the peer review process unethical? 4.Does the peer review process enforce orthodoxy? 6

Peer Review How to conduct a peer review: Is the research novel? – Literature review is needed – Sometimes similar results are published simultaneously – Novelty is a gray area What about dissertations? What about foreign-language publications? 7

Peer Review How to conduct a peer review: Is the paper clear and logical? – Is there enough detail? Is there too much detail? – Is the research reproducible? 8

Peer Review How to conduct a peer review: Is the research significant? – Is it more than just an engineering exercise? – Is it important for researchers in the field? – Is it interesting? – Is it non-obvious? – These are difficult questions because they entail value judgements 9

How to conduct a peer review: What is the contribution of the paper? Is the contribution clearly stated in the abstract, introduction, and conclusion? Are the claims supported in the paper? Different types of contributions New theory New synthesis New application Tutorial 10 Peer Review

How to conduct a peer review: Recommend for or against publication What are the standards of the journal / conference? Do you recommend a different publication venue? Revision (major, minor)? Resubmission? Is another review needed? Justify your review with comments Constructive criticism General comments Specific comments For the editor: How confident are you in your review? 11

Peer Review How to conduct a peer review: Should the qualifications (good or bad) of the author be considered? You need to find a balance between overly permissive and overly restrictive Consider the standards of the target publication Students tend to be overly permissive in reviews Be diplomatic in your criticism 12

Peer Review How to conduct a peer review: Ethical Issues If you submit, then you should review Do not review a paper if you are not qualified Make sure you disclose any conflict of interest Prior publication (conference vs. journal) Simultaneous submission Plagiarism Submitted material is for review only unless the author allows its use for other purposes Do unto others as you would have them do unto you 13

Peer Review Outline 1.Overview 2.How to conduct a peer review 3.Is the peer review process unethical? 4.Does the peer review process enforce orthodoxy? 14

Peer Review Is the peer review process ethical? Two different types of peer review: Blind Double Blind Most peer reviews are Blind – a few are Double Blind Proposition: An institution is unethical if it enables and protects unethical actions 15

Peer Review A reviewer was incompetent62% A reviewer was biased51% A reviewer required unnecessary references to his/her publications23% Comments from reviewer included personal attacks18% A reviewer delayed the review so he could publish an article on the same topic10% A reviewer breached confidentiality7% A reviewer used your material without your permission5% 16 D. Resnik, C. Guiterrez-Ford, and S. Peddada, "Perceptions of Ethical Problems with Scientific Journal Peer Review: An Exploratory Study,” 2008

Peer Review Who is peer reviewing your papers? Not collaborators – they are prevented due to conflict of interest Competitors The peer review process allows your competitors to anonymously undermine your work The anonymity of peer reviewers is protected The process assumes that peer reviewers will act ethically An institution is unethical if it enables and protects unethical actions 17

Peer Review What features of peer review make it ethical or unethical? Single blind review? Double blind review? Single blind partially open review? (Philica) Open review? Reverse single blind review? 18

Peer Review Outline 1.Overview 2.How to conduct a peer review 3.Is the peer review process unethical? 4.Does the peer review process enforce orthodoxy? 19

Peer Review Does peer review enforce orthodoxy? Rosalyn Yalow, 1977 Nobel Prize in Physiology or Medicine: “In 1955 we submitted the paper to Science. … The paper was held there for eight months before it was reviewed. It was finally rejected. We submitted it to the Journal of Clinical Investigations, which also rejected it.” 20

Peer Review Does peer review enforce orthodoxy? Mitchell Feigenbaum, pioneer of chaos theory: “Both papers were rejected, the first after a half-year delay. By then, in 1977, over a thousand copies of the first preprint had been shipped. This has been my full experience. Papers on established subjects are immediately accepted. Every novel paper of mine, without exception, has been rejected by the refereeing process. The reader can easily gather that I regard this entire process as a false guardian and wastefully dishonest.” 21

Peer Review Does peer review enforce orthodoxy? Tuzo Wilson, developed the theory of Hawaiian island formation: “I … sent [my paper] to the Journal of Geophysical Research. They turned it down. … They said my paper had no mathematics in it, no new data, and that it didn’t agree with the current views. Therefore, it must be no good. Apparently, whether one gets turned down or not depends largely on the reviewer. The editors, too, if they don’t see it your way, or if they think it’s something unusual, may turn it down.” 22

Peer Review Does peer review enforce orthodoxy? Stephen Hawking John Bardeen, two-time Nobel prize winner Theodore Maiman, inventor of the laser … and many others … 23

Peer Review Why does peer review enforce orthodoxy? Dilution of research talent In 1900, 10% of physicists were nominated for Nobel prize The number of physicists has increased by a factor of 1,000 University dependence on grant funding (overhead costs about 50%) 24

References “The Task of the Referee,” by Alan Jay Smith “A Student’s Guide to Peer Review,” by Dennis Bernstein “Is peer review unethical?” by Valentine Cawley 25