1. Archived File The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the OER Public Archive Home Page for more details about archived files.archivedOER Public Archive Home Page

2. Improving Peer Review CSR Initiatives toni scarpa scarpat@csr.nih.gov 301-435-1109 PRAC Nov. 3, 2008 National Institutes of Health U.S. Department of Health and Human Services

3.  New Data and the Drivers for Change  CSR’s Efforts to Enhance Peer Review  The NIH Director’s Peer Review Initiatives Improving Peer Review

4. New Data and the Drivers for Change

5. CSR Peer Review: 2008 77,000 applications received 56,000 applications reviewed 16,000 reviewers 240 Scientific Review Officers 1,800 review meetings

6. $13.7 $15.6 $17.8 $20.5 $23.3 $27.1 $28.0 $28.6 $29.1 $29.5 $0 $5 $10 $15 $20 $25 $30 1998199920002001 2002200320042005200620072008 2009 1st Driver: The NIH Budget Doubling $ In Billions

7. 2 nd Driver: Number of Applications Submitted Historical Growth

8. SBIR Applications 7000 6000 5000 4000 3000 2000 1000 0 199920002001200220032004200520062007 INST CSR

9. R01 Applications Reviewed at CSR vs. Other ICs

10. R21 Applications Reviewed at CSR vs. Other ICs

11. 3 rd Driver: Reviewer’s Load Applications Per Reviewer October Council Rounds

12. 4th Driver: CSR Budget $ Millions

13. 4th Driver: CSR Budget $ Millions

14. Annual Savings in Reviewers’ Expenses Budget Non-refundable tickets with one possible change $15 million 3,000 fewer reviewers $3 million 15% reviews using electronic platforms $5 million One meeting a year on the West Coast $1.8 million

15. Next Savings? Replace shipping of CDs with online downloading Cost of 18,000 CDs $750,000 Cost of shipping$200,000 Handling $50,000 5% more reviews using electronic platform SREA budget $1,500,000 CSR budget -$100,000 More meetings in Baltimore Savings/100 meetings$500,000 Review of fellowship applications

16. CSR’s Efforts to Enhance Peer Review

17. 1.Reorganizing CSR’s Review Groups 2.Recruiting CSR Staff 3.Shortening the Review Cycle 4.Holding Out-of-Town Meetings 5.Recruiting the Best Reviewers 6.Performing a Second Cycle of IRG Reviews 7.Revising Study Section Guidelines 8.Training Chairs 9.Managing Conflicts of Interest 10.Advancing Additional Review Platforms and Processes CSR’s Efforts to Enhance Peer Review

18. 1.Reorganizing CSR’s Review Groups

19. 1. CSR Organization of Peer Review DivisionsScientific Review OfficersIntegrated Review Groups

20. 1. CSR Reorganization Rene Etcheberrigaray Translational and Clinical Sciences Cardiovascular and Respiratory Sciences Surgical Sciences, Biomedical Imaging and Bioengineering Musculoskeletal, Oral And Skin Sciences Oncology 2 – Translational Clinical Vascular and Hematology Physiological and Pathological Sciences Endocrinology, Metabolism, Nutrition & Reproductive Sciences Immunology Infectious Diseases & Microbiology Digestive, Kidney & Urological Systems Neuroscience, Development and Aging Brain Disorders & Clinical Neuroscience Molecular, Cellular & Developmental Neuroscience Integrative, Functional & Cognitive Neuroscience Emerging Technologies & Training in Neuroscience Biology of Development & Biobehavioral & Behavioral Processes Risk, Prevention& Health Behaviors Epidemiology & Population Sciences Healthcare Delivery & Methodologies AIDS & Related Research AIDS, Behavioral and Population Sciences Basic and Integrative Biological Sciences Biological Chemistry & Macromolecular Biophysics Bioengineering Sciences & Technologies Genes, Genomes &Genetics Oncology 1 – Basic Translational Cell Biology Interdisciplinary Molecular & Training Don Schneider

21. Update on CSR Realignments Don Schneider Update on CSR Realignments Don Schneider 1. CSR Organization of Peer Review

22. CSR’s Efforts to Enhance Peer Review 1.Reorganizing CSR’s Review Groups 2.Recruiting CSR Staff

23. 3 Division Directors 6 Integrated Review Chiefs 20 Scientific Review Officers

24. CSR’s Efforts to Enhance Peer Review 1.Reorganizing CSR’s Review Groups 2.Recruiting CSR Staff 3.Shortening the Review Cycle

25. 3. Shortening the Review Cycle The Goal To provide applicants a review and score within 3 months of application submission. This will permit resubmission of applications (when doable and desirable) 4 months earlier than in the past.

26. RO1 A1 Resubmission Within 4 Months of Original Application

27. CSR’s Efforts to Enhance Peer Review 1.Reorganizing CSR’s Review Groups 2.Recruiting CSR Staff 3.Shortening the Review Cycle 4.Holding Out-of-Town-Meetings

28. 4. Holding Out-of-Town Meetings To facilitate participation from colleagues on the West Coast, one meeting out of three each year will be held in the metro areas of: o Los Angeles o San Francisco o Seattle o Chicago Benefits Great satisfaction Significant savings Ranking of hotels

29. 5. Near-Term Solutions for Recruiting and Retaining the Best Reviewers Move a meeting a year to the West Coast Develop a national registry of volunteer reviewers Searchable database with 3,500 reviewers Provide tangible rewards for reviewers No submission deadlines for chartered members of study sections (effective February 2008). 1,574 chartered members used flexible deadlines during the last 6 months Provide flexible time for reviewers Choice of 3 times/year for 4 years or 2 times/year for 6 years

30. CSR’s Efforts to Enhance Peer Review 1.Reorganizing CSR’s Review Groups 2.Recruiting CSR Staff 3.Shortening the Review Cycle 4.Out-of-Town Meetings 5.Recruiting the Best Reviewers 6.Performing a Second Cycle of IRG Reviews

31. 6. Second Cycle of IRG Reviews Scheduled 2008-5 IRGsScheduled 2009-12 IRGs (Continued) Brain Disorders and Clinical Neuroscience (BDCN) and Molecular, Cellular and Developmental Neuroscience (MDCN) Bioengineering Sciences and Technologies (BST) andSurgical Sciences, Biomedical Imaging and Bioengineering (SBIB) Biology of Development and Aging (BDA), Emerging Technologies and Training in Neuroscience (ETTN) andIntegrative, Functional and Cognitive Neuroscience (IFCN) AIDS and Related Research (AARR) and Endocrinology, Metabolism, Nutrition and Reproductive Sciences (EMNR) Scheduled 2009-12 IRGsScheduled 2010-8 IRGs Biobehavioral and Behavioral Processes (BBB) and Risk, Prevention and Health Behavior (RPHB) Vascular and Hematology (VH) and Cardiovascular and Respiratory Sciences (CVR) Healthcare Delivery and Methodologies (HDM) and Epidemiology and Population Sciences (EPS) Musculoskeletal, Oral and Skin Sciences (MOSS) and Digestive, Kidney and Urological Systems (DKUS) Immunology (IMM) and Infectious Diseases and Microbiology (IDM) Genes, Genomes and Genetics (GGG) and Biological, Chemical and Macromolecular Biophysics (BCMB) Oncology 1 — Basic and Translational (OBT) and Oncology 2 — Translational Clinical (OTC) Interdisciplinary Molecular Sciences and Training (IMST) and Cell Biology (CB)

32. 6. Performing a Second Cycle of IRG Reviews All pertinent study section chairs were invited Advice on study section and IRG Revision of study sections description/outline Training of the chair

33. 6. Performing a Second Cycle of IRG Reviews Second Cycle of IRG Reviews Don Schneider

34. CSR’s Efforts to Enhance Peer Review 1.Reorganizing CSR’s Review Groups 2.Recruiting CSR Staff 3.Shortening the Review Cycle 4.Holding Out-of-Town Meetings 5.Recruiting the Best Reviewers 6.Performing a Second Cycle of IRG Reviews 7.Revising Study Section Guidelines

35. 7. Revising Study Section Outlines Cellular Signaling and Regulatory Systems [Roster] The Cellular Signaling and Regulatory Systems (CSRS) study section reviews applications that focus on the initiation and execution of programs that control cellular homeostasis and physiology. A distinguishing characteristic of these applications is an emphasis on signaling networks and the coordination of processes related to cell proliferation, survival, and growth. Cell cycle regulation, mitosis, meiosis, checkpoint controls and regulation by ubiquitination Proteolytic mechanisms associated with cell cycle, senescence and death Programmed cell death and apoptosis, particularly their regulation in the context of stress, growth, and transformation. Proliferation and growth control by the nucleus; signaling pathways regulating transcription Integrative cell physiology, e.g., stress, clocks, cellular modeling; cell differentiation and transformation Basic studies of cytokine signaling Application of state-of-the-art technologies such as imaging and computational modeling of cellular signaling networks Study sections with most closely related areas of similar science listed in rank order are: Molecular and Integrative Signal Transduction Intercellular Interactions Membrane Biology and Protein Processing Molecular Genetics A Molecular Genetics B

36. CSR’s Efforts to Enhance Peer Review 1.Reorganizing CSR’s Review Groups 2.Recruiting CSR Staff 3.Shortening the Review Cycle 4.Holding Out-of-Town Meetings 5.Recruiting the Best Reviewers 6.Performing a Second Cycle of IRG Reviews 7.Revising Study Section Guidelines 8.Training Chairs

37. 8. Training Study Section Chairs Scope and Schedule 120 new chairs every year, more if “permanent” SEPs chairs are included 5 meetings in January 2009 for chairs recently appointed 5 meetings in July 2009 for chairs appointed next June Training Program Share data Explain the new changes and the significance Share the best practice (ours and theirs) Answer questions and address concerns Make Chairs more effective stakeholders

38. CSR’s Efforts to Enhance Peer Review 1.Reorganizing CSR’s Review Groups 2.Recruiting CSR Staff 3.Shortening the Review Cycle 4.Holding Out-of-Town Meetings 5.Recruiting the Best Reviewers 6.Performing a Second Cycle of IRG Reviews 7.Revised Study Section Guidelines 8.Training of Chairs 9.Managing Conflicts of Interest

39. Report of Conflict of Interest Committee Noni Byrnes

40. CSR’s Efforts to Enhance Peer Review 1.Reorganizing CSR’s Review Groups 2.Recruiting CSR Staff 3.Shortening the Review Cycle 4.Holding Out-of-Town Meetings 5.Recruiting Reviewers 6.Performing a Second Cycle of IRG Review 7.Revising Study Section Guidelines 8.Training of Chairs 9.Managing Conflicts of interest 10.Advancing Additional Review Platforms and Processes

41. 10. Advancing Additional Review Platforms and Process Video (VED) Internet (AED) Editorial Board Order of Discussion

42. 10. Advancing Additional Review Platforms and Processes Editorial Board Review oHeavy triage based on innovation and potential science transformation by a small study section of distinguished, broad-science reviewers (the editors) oSpecific science reviewed by appropriate reviewers (the editorial board) oFinal ranking by the editors

43. 10. Advancing Additional Review Platforms and Process New Transformative R01 Betsy Wilder

44. 10. Advancing Additional Review Platforms and Process Results of Asynchronous Electronic Discussion Survey Andrea Kopstein

45. NIH Enhancement of Peer Review The Charge from Dr. Zerhouni: “Fund the best science, by the best scientists, with the least administrative burden…” with the least administrative burden…” http://enhancing-peer-review.nih.gov Two advisory committees to the NIH Director

46. The NIH Director’s Peer Review Initiatives

47. Corporate NIH Peer Review Enhancement: Implementation Raynard Kington

48. This is CSR September 2008

49. Recognizing Outstanding Reviewers Going above and beyond the call of duty Having a significant impact on the quality of peer review Educating and inspiring other reviewers Setting the highest standard of peer review excellence

50. The First Marcy Speer Award, 2007 Marcy Speer

51. 2008 Marcy Speer Outstanding CSR Reviewer Award Dr. David J. Sahn, M.D. Oregon Health & Science University

52. The 2 nd Marcy Speer Award, 2008