Presentation is loading. Please wait.

Presentation is loading. Please wait.

The Review Process: Responsibilities and Rewards Patrick J. Flynn Associate Professor

Similar presentations


Presentation on theme: "The Review Process: Responsibilities and Rewards Patrick J. Flynn Associate Professor"— Presentation transcript:

1 The Review Process: Responsibilities and Rewards Patrick J. Flynn Associate Professor

2 Outline Introduction and Definitions Introduction and Definitions Reviewing: what to do, what not to do, etc. Reviewing: what to do, what not to do, etc. Examples and anecdotes Examples and anecdotes Plagiarism: definitions Plagiarism: definitions Avoiding plagiarism: rules for writers Avoiding plagiarism: rules for writers Examples and anecdotes Examples and anecdotes Conclusions Conclusions

3 Goals Discuss (interactively) an infrequently-discussed professional duty and its details Discuss (interactively) an infrequently-discussed professional duty and its details For your own benefit as an author, understand the review process For your own benefit as an author, understand the review process For the community’s benefit (from quality reviews), discuss expectations of reviewers For the community’s benefit (from quality reviews), discuss expectations of reviewers For society’s benefit, reinforce the traditional negative attitude toward plagiarism For society’s benefit, reinforce the traditional negative attitude toward plagiarism

4 Introduction Peer Review: a “community assessment” of quality Peer Review: a “community assessment” of quality  of papers presented at meetings  of papers submitted to the archival literature  of proposals submitted for funding  of the “body of work” of a professor seeking tenure and/or promotion

5 Introduction (ctd.) Peer review Peer review  Important  Fair (no “guaranteed publication” because of who you are)  Expensive  Time of reviewers  Time of editorial staff and volunteers  Confidential (usually; depends on venue)

6 Editorial Flow in a typical journal 1. Author submits paper for review 2. Editorial staff logs the paper and sends it to the Editor (Editor-in-Chief, etc.) 3. Editor sends it to an Associate Editor (AE) 4. AE identifies reviewers and tells staff to send paper to reviewers 5. Several months go by

7 Editorial Flow (ctd.) 7. AE assembles reviews and decides: Accept (rare) Accept (rare) Minor revision (examined by AE) Minor revision (examined by AE) Major revision (re-reviewed by same reviewers) Major revision (re-reviewed by same reviewers) Reject Reject 8. Inform author of decision

8 Editorial Flow (ctd.) 9. Await revised manuscript (several months) and statement of changes made in response to reviews 10. AE examines revision and (if applicable) obtains new reviews (several months) 11. Decision is made; accepted paper scheduled for publication

9 Editorial Flow Problems Time delay (biggest problem) Time delay (biggest problem)  Unresponsive AEs or EICs  Unresponsive reviewers Poor quality reviews (short and meaningless, biased, hypercritical, incompetent, …) Poor quality reviews (short and meaningless, biased, hypercritical, incompetent, …) Author complaints (often justified, sometimes not) Author complaints (often justified, sometimes not)

10 Referee Duties Handout: “The Task of the Referee”, A. Smith, IEEE Computer, April 1990, pp Handout: “The Task of the Referee”, A. Smith, IEEE Computer, April 1990, pp Determine the paper’s ‘contribution’ Determine the paper’s ‘contribution’  Goal  Scope  Potential impact

11 Referee Duties (ctd.) Check accuracy and validity Check accuracy and validity Determine appropriateness to venue Determine appropriateness to venue Assess quality of presentation Assess quality of presentation Spot instances of plagiarism or poor citation quality Spot instances of plagiarism or poor citation quality If paper is not publishable as is, but is salvageable, determine where and how improvements can be made. If paper is not publishable as is, but is salvageable, determine where and how improvements can be made.

12 Referee duties Write a Defensible Review Write a Defensible Review  Summary of paper  Assessment of appropriateness, correctness, impact, and presentation  Recommendation: accept, revise, reject  Suggestions for improvement (citations to add, details, etc.)  Don’t use your position as reviewer to force unnecessary citations to your own work

13 Example Reviews See handout See handout First: IEEE Transactions review (negative) First: IEEE Transactions review (negative) Second: NSF (medium) Second: NSF (medium) Third: IEEE Transactions review (lukewarm) Third: IEEE Transactions review (lukewarm) Most reviews are “medium” (few strongly negative, few strongly positive) Most reviews are “medium” (few strongly negative, few strongly positive)

14 Re-use and fair use See Stone’s paper in the handouts See Stone’s paper in the handouts Citations: always required unless prior work was not published anywhere in any form Citations: always required unless prior work was not published anywhere in any form Sometimes, even previously unpublished work is cited (it never hurts to cite) Sometimes, even previously unpublished work is cited (it never hurts to cite) Generally, redundant citations (e.g., to conference paper and a follow-on journal paper) aren’t warranted Generally, redundant citations (e.g., to conference paper and a follow-on journal paper) aren’t warranted Self-citations should be kept to a minimum (avoid gratuitous self-citations) Self-citations should be kept to a minimum (avoid gratuitous self-citations)

15 Re-use and fair use (ctd.) Quotations Quotations  Short (fair use): cite and quote  Long: cite, inset or quote, obtain permission  “Almost verbatim”: cite, inset or quote, obtain permission  Reuse of artwork: cite, inset or quote, obtain permission Permissions obtained from copyright holder Permissions obtained from copyright holder

16 Copyrights If your work is published, you assign (transfer) copyright to the publisher If your work is published, you assign (transfer) copyright to the publisher You cannot then give permission for re-use. You cannot then give permission for re-use. Handout: copyright form (note: revised to warrant compliance with the DMCA) Handout: copyright form (note: revised to warrant compliance with the DMCA) Handout: IEEE re-use permission form (routinely granted) Handout: IEEE re-use permission form (routinely granted)

17 Plagiarism and other unprofessional acts Re-use without citation Re-use without citation Multiple submissions Multiple submissions Theft and submission/publication of others’ ideas Theft and submission/publication of others’ ideas

18 Re-use without citation Handout: Lai/Sethi incident Handout: Lai/Sethi incident  Sethi’s complaint  Ledley’s response  The paper in question  The relevant pages from Sethi’s TPAMI paper

19 Re-use ctd. Comments Comments  Plagiarized amount: 2-3 paragraphs (not much)  It was still plagiarism, even though the source was cited.  More than fair use

20 Another example Handout: Editorial from Bob Ledley Handout: Editorial from Bob Ledley Hsia and Huang, Pattern Recognition 26: Hsia and Huang, Pattern Recognition 26: Two pages taken from Kriegman and Ponce (IJCV 5: ) Two pages taken from Kriegman and Ponce (IJCV 5: )

21 But here’s another problem Insufficient number of reviewers Insufficient number of reviewers Pattern Recognition: only one review per submission Pattern Recognition: only one review per submission Plagiarism hence easier to miss Plagiarism hence easier to miss

22 But it’s less important Authors should be professional in citing other work Authors should be professional in citing other work  It’s expected  It’s required by law and convention  It speaks to reputation (very important to young researchers)

23 Another example TPAMI submission, from a previously unknown author with no academic or industrial affiliation TPAMI submission, from a previously unknown author with no academic or industrial affiliation Discovered to be a photocopy of a technical report from a leading researcher (with a new cover page) Discovered to be a photocopy of a technical report from a leading researcher (with a new cover page) PAMI “death penalty” (no future submission will be accepted) PAMI “death penalty” (no future submission will be accepted) Scary: the paper was out for review, but the cognizant AE spotted the problem Scary: the paper was out for review, but the cognizant AE spotted the problem

24 Multiple Submissions Handout: IEEE TPAMI editorial (Bowyer and Flynn) Handout: IEEE TPAMI editorial (Bowyer and Flynn) The same paper was submitted administratively to TPAMI and IVC The same paper was submitted administratively to TPAMI and IVC Front 40% identical Front 40% identical Remainder very similar Remainder very similar Rejected from TPAMI without review Rejected from TPAMI without review

25 Multiple submissions (ctd.) Another case: paper submitted to at least four journals (including PAMI) simultaneously (PJF got it twice) Another case: paper submitted to at least four journals (including PAMI) simultaneously (PJF got it twice) Rejected without review Rejected without review

26 Multiple Submissions: what’s the problem? “Let’s allow it! Whom does it hurt?” “Let’s allow it! Whom does it hurt?”  Reviewers - burden increases  Prospective copyright holders  Reviewers

27 Conclusions Recall the goals Recall the goals  Sensitivity to the review process for your and its benefit  Understanding citation expectations for professional writings  Understanding legalities of copyright  Make you more comfortable with the “trivia” associated with writing

28 Conclusions Writing is difficult Writing is difficult Reviewing is difficult Reviewing is difficult Organizing journal issues and conferences is difficult Organizing journal issues and conferences is difficult What makes it worthwhile is the knowledge gain they promote What makes it worthwhile is the knowledge gain they promote You are expected to support that You are expected to support that


Download ppt "The Review Process: Responsibilities and Rewards Patrick J. Flynn Associate Professor"

Similar presentations


Ads by Google