1. Vu Pham Refereeing and Discussant Guidelines Susan Godlonton AGRODEP AIEN III Workshop Dakar, Senegal 4 th June, 2014

2. Vu Pham Components of a referee report: Cover letter Summary Key concerns Decision Report Overall view Main concerns Smaller concerns No “one-size-fits-all approach” Referee Report

3. Vu Pham What is a good paper? Clear Coherent (understandable) Correct Additional contribution Value added of answer to question Credibility of identification strategy Validity and robustness of approach Referee Report: Good paper?

4. Vu Pham Don’t repeat the abstract, the editor can read the abstract How would you summarize the paper What do you think is the key contribution Do you think the authors address it correctly Referee Report: Overall view

5. Vu Pham Key issues that the author needs to address Ensure that it is clear which of your concerns are fundamental Don’t ask the author to write a different paper Be constructive! Try to provide solutions where possible accounting for data limitations Referee Report: Main Concerns

6. Vu Pham Related literature Inappropriate to the actual material in the paper Inaccurately described Incomplete (very common) Logical argument Incorrect application of economic concepts Inaccurate mathematical derivations Loose ties between the economic model and the empirical analysis Referee Report: Main concerns

7. Vu Pham Econometric tools are being used inappropriately Incorrectly used Assumptions do not hold Assumptions not defended with evidence Estimated key coefficients of interest Not properly identified Not robust Additional robustness checks needed Interpretation of empirical results is inappropriate Overstates the contribution Overstates the claim (e.g. causation vs. correlation) Conclusions are incorrectly made or expressed Referee Report: Main concerns

8. Vu Pham Econometric tools are being used inappropriately Incorrectly used Assumptions do not hold Assumptions not defended with evidence Estimated key coefficients of interest Not properly identified Not robust Additional robustness checks needed Interpretation of empirical results is inappropriate Overstates the contribution Overstates the claim (e.g. causation vs. correlation) Conclusions are incorrectly made or expressed Referee Report: Main concerns

9. Vu Pham Do not bring these up when discussing a paper Specific places in text where discussion hard to follow or confusing Not using standard notation Spelling and grammatical errors Missing data sources and poorly constructed Tables or Figures References to the literature that are missing or incorrect Referee Report: Minor concerns

10. Vu Pham Refereeing http://blogs.worldbank.org/impactevaluations/how-much-to-referee- and-how-to-do-it http://blogs.worldbank.org/impactevaluations/how-much-to-referee- and-how-to-do-it http://marcfbellemare.com/wordpress/2012/01/contributing-to-public- goods-my-20-rules-for-refereeing/ http://marcfbellemare.com/wordpress/2012/01/contributing-to-public- goods-my-20-rules-for-refereeing/ http://marginalrevolution.com/marginalrevolution/2006/10/how_to_b e_a_goo.html http://marginalrevolution.com/marginalrevolution/2006/10/how_to_b e_a_goo.html Discussant http://chrisblattman.com/2010/02/22/the-discussants-art/ Other resources will be uploaded to the AIEN workshop Additional Resources

11. Vu Pham Each been assigned a paper/set of slides from one of your peers Focus Clearly define the research question Define and discuss the identification strategy Propose alternative credible identification strategy For today…

12. Vu Pham Focus on the identification strategy What is the identifying variation? What assumptions are made by using the chosen identification strategy (implicitly/explicitly)? Do they make sense? To what extent are these true? How else can the author test the assumptions? What robustness checks or placebo tests could the author run? What would be a more preferable strategy? For today…

13. Vu Pham Thinking about Identification Homework Assignment

14. Vu Pham Group into: Health Education Agriculture Discuss proposed ideas, provide feedback to one another Thinking about Identification Exercise