1. An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association for Thoracic Surgery March 4, 2011 National Institutes of Health U.S. Department of Health and Human Services

2. National Institutes of Health National Institute on Alcohol Abuse and Alcoholism National Institute on Alcohol Abuse and Alcoholism National Institute of Arthritis and Musculoskeletal and Skin Diseases National Institute of Arthritis and Musculoskeletal and Skin Diseases National Cancer Institute National Cancer Institute National Institute on Drug Abuse National Institute on Drug Abuse National Institute of Environmental Health Sciences National Institute of Environmental Health Sciences National Institute on Aging National Institute on Aging National Institute of Child Health and Human Development National Institute of Child Health and Human Development National Institute on Deafness and Other Communication Disorders National Institute on Deafness and Other Communication Disorders National Eye Institute National Eye Institute National Human Genome Research Institute National Human Genome Research Institute National Institute of Mental Health National Institute of Mental Health National Institute of Neurological Disorders and Stroke National Institute of Neurological Disorders and Stroke National Institute of General Medical Sciences National Institute of General Medical Sciences National Institute of Nursing Research National Institute of Nursing Research National Library of Medicine National Library of Medicine Center for Scientific Review Center for Scientific Review National Center for Complementary and Alternative Medicine National Center for Complementary and Alternative Medicine National Institute of Allergy and Infectious Diseases National Institute of Allergy and Infectious Diseases Fogarty International Center Fogarty International Center National Center for Research Resources National Center for Research Resources Clinical Center Clinical Center National Center on Minority Health and Health Disparities National Center on Minority Health and Health Disparities National Institute of Biomedical Imaging and Bioengineering National Institute of Biomedical Imaging and Bioengineering Office of the Director Center for Information Technology Center for Information Technology National Heart, Lung, and Blood Institute National Heart, Lung, and Blood Institute National Institute of Dental and Craniofacial Research National Institute of Dental and Craniofacial Research National Institute of Diabetes and Digestive and Kidney Diseases National Institute of Diabetes and Digestive and Kidney Diseases

3. National Institutes of Health Center for Scientific Review Study Section Institute Advisory Councils and Boards Institute Director Assigns to IC & I IRG/Study Section Reviews for Scientific Merit Evaluates for Relevance Recommends Action Takes Final Action Initiates Research Idea Conducts Research Allocates Funds Submits Application Review Process for a Research Grant Research Grant Application School or Research Center

4. Scientific Review Process Dual Review System for Grant Applications Second Level of Review NIH Institute/Center Council NIH Institute/Center Council Second Level of Review NIH Institute/Center Council NIH Institute/Center Council First Level of Review CSR or Institute Review Scientific Review Group (Study Section) First Level of Review CSR or Institute Review Scientific Review Group (Study Section)

5. Receives all NIH applications Refers them to NIH Institutes/Centers and to scientific review groups Reviews majority of grant applications for scientific merit Your Application Goes to the NIH Center for Scientific Review (CSR) Focal Point for Initial Review at NIH

6. Translational and Clinical Sciences Translational and Clinical Sciences Cardiovascular and Respiratory Sciences Cardiovascular and Respiratory Sciences Surgical Sciences, Biomedical Imaging and Bioengineering Surgical Sciences, Biomedical Imaging and Bioengineering Musculoskeletal, Oral And Skin Sciences Musculoskeletal, Oral And Skin Sciences Oncology 2 – Translational Clinical Oncology 2 – Translational Clinical Vascular and Hematology Hematology Physiological and Pathological Sciences Physiological and Pathological Sciences Endocrinology,Metabolism, Nutrition & Reproductive Sciences Reproductive Sciences Endocrinology,Metabolism, Nutrition & Reproductive Sciences Reproductive Sciences Immunology Immunology Infectious Diseases & Microbiology Infectious Diseases & Microbiology Digestive, Kidney & Urological Systems Digestive, Kidney & Urological Systems Neuroscience, Development and Aging Neuroscience, Development and Aging Brain Disorders & Clinical Neuroscience Brain Disorders & Clinical Neuroscience Molecular, Cellular & Developmental Neuroscienc Developmental Neuroscienc e Molecular, Cellular & Developmental Neuroscienc Developmental Neuroscienc e Integrative, Functional & Cognitive Neuroscience Integrative, Functional & Cognitive Neuroscience Emerging Technologies & Training in Neuroscience Emerging Technologies & Training in Neuroscience Biology of Development and Aging Biology of Development and Aging Biobehavioral & Biobehavioral & Behavioral Processes Behavioral Processes Biobehavioral & Biobehavioral & Behavioral Processes Behavioral Processes Risk, Prevention& Health Behaviors Risk, Prevention& Health Behaviors Epidemiology & Population Sciences Epidemiology & Population Sciences Healthcare Delivery & Methodologies Healthcare Delivery & Methodologies AIDS & Related Research AIDS & Related Research AIDS, Behavioral and Population Science s AIDS, Behavioral and Population Science s Basic and Integrative Biological Sciences Basic and Integrative Biological Sciences Biological Chemistry & MacromolecularBiophysics MacromolecularBiophysics Bioengineering Sciences & Technologies Bioengineering Sciences & Technologies Genes, Genomes & Genetics Genes, Genomes & Genetics Oncology 1 – Basic Translational Translational Cell Biology InterdisciplinaryMolecular & Training InterdisciplinaryMolecular & Training CSR Review Divisions

7. Help Get Your Application to the Right Study Section Review CSR Integrated Review Group and Scientific Review Group (Study Section) guidelines to identify a home for your application. Submit a Cover Letter!

8. Help Get Your Application to the Right Study Section http://www.csr.nih.gov/

9. Help Get Your Application to the Right Study Section Integrated Review Group

10. Help Get Your Application to the Right Study Section Study Section

11. Cardiovascular and Respiratory Sciences IRG Cardiac Contractility, Hypertrophy and Failure Cardiovascular Differentiation and Development (CDD) Clinical and Integrative Cardiovascular Sciences (CICS) Electrical Signaling, Ion Transport and Arrhythmias (ESTA) Myocardial Ischemia and Metabolism (MIM) Lung Injury, Repair and Remodeling (LIRR) Lung Cellular, Molecular and Immunobiology Respiratory Integrative Biology and Translational Research (RIBT)

12. Surgical, Biomedical Imaging and Bioengineering IRG Clinical, Molecular Imaging and Probe Development (CMIP) Medical Imaging (MEDI) Surgery, Anesthesiology and Trauma (SAT) Bioengineering, Technology and Surgical Sciences (BTSS) Biomedical Imaging Technology A & B (BMIT)

13. Division of Translational and Clinical Sciences (DTCS) Trans-IRG Imaging SEP Goal - enhance review of multidisciplinary, translational research with a focus on imaging Transformative in nature (translational research) Include humans Use imaging as a measure of diagnosis, intervention or treatment success/efficacy Editorial Board (Two Stage) Review First stage – 3 mail reviews (technical focus) Second stage – 3 clinical impact reviews Crosses disciplines of cancer, cardiovascular, musculoskeletal and neuroscience

14. Submit a Cover Letter The cover letter should be used for a number of important purposes: Suggest Institute/Center assignment Suggest review assignment Identify individuals in conflict Identify areas of expertise needed to evaluate the application Discuss any special situations It is NOT appropriate to use the cover letter to suggest specific reviewers.

15. Peer Review in CSR CSR Study Sections are managed by a Scientific Review Officer (SRO) who is a doctoral-level professional, whose scientific background is close to the focus of the study section. Each CSR standing study section has 12-25 regular members who are from the scientific community. Temporary members are recruited as needed. About 60-100 applications are normally reviewed at each study section meeting.

16. The Role of Your Scientific Review Officer Designated Federal Official responsible for the overall review process

17. Your Study Section Meeting

18. Your Application Is Reviewed In Order Clustered and Ordered for Fairness New and Early Stage Investigator applications are usually reviewed first Applications in each cluster are reviewed in order of their preliminary scores

19. New Investigator (NI):New Investigator (NI): PD/PI who has not yet competed successfully for a substantial NIH research grant oFor multiple PD/PIs-all PD/PIs must meet requirements for NI status Early Stage Investigator (ESI):Early Stage Investigator (ESI): PD/PI who qualifies as a New Investigator AND is within 10 years of completing the terminal research degree or is within 10 years of completing medical residency (or equivalent)

20. Reviewers discuss about 50- 60% of the applications The panel will discuss any application a reviewer wants to discuss Discussions Focus on the Best Applications

21. Additional Review Platforms Helps to recruit reviewers Electronic review modes reduce travel Electronic Reviews Telephone Assisted Meetings Video Assisted Meetings Telepresence Internet Assisted Meetings Editorial Board Review – multidisciplinary

22. Review Criteria Overall Impact: exert a sustained, powerful influence on the research field(s) involved Assessment of the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved Core Criteria Significance Investigator(s) Innovation Approach Environment oReview criteria each scored from 1-9

23. How Your Application Is Aligned to the Review CriteriaApplication Section Significance Research Strategy a. Significance Investigator(s) Biosketch Personal Statement Innovation Research Strategy b. Innovation Approach Research Strategy c. Approach Environment Resources Environment

24. 9-Point Scoring Scale ImpactScoreDescriptor High Impact 1Exceptional 2Outstanding 3Excellent Medium Impact 4Very Good 5Good 6Satisfactory Low Impact 7Fair 8Marginal 9Poor

25. Scores for each review criterion Essentially unedited critiques Administrative notes if any If your application is discussed, you also will receive: An overall impact/priority score and percentile ranking An Summary of review discussion Budget recommendations Your Summary Statement

26. Summary Statement New Indicator for Early Stage Investigators Percentile: 29 Percentile in whole numbers Impact/Priority Score 10-90 range Program Officer

27. Read instructions Never assume that reviewers will know what you mean Refer to literature thoroughly State rationale of proposed investigation Include well-designed tables and figures Present an organized, lucid write-up Obtain pre-review from faculty at your institution NIH Grant Writing Tips http://grants.nih.gov/grants/grant_tips.htm When Preparing an Application

28. Additional Factors to Consider Focus on significance/impact Make it exciting Be very clear Do not assume too much Have realistic aims and timelines -- Don’t be too ambitious Be brief with things that everybody knows Note the study’s limitations Proofread the application

29. Who Can Answer Your Questions? Before You Submit Your Application A Program Officer at an NIH Institute or Center After You Submit Your Scientific Review Officer After Your Review Your Assigned Program Officer

30. Key NIH Review and Grants Web Sites NIH Center for Scientific Review http://www.csr.nih.gov NIH Office of Extramural Research http://grants.nih.gov/

31. Helpful Handouts Insiders Guide What Happens to NIH Grant Application to Peer Review Your Grant Application Useful Web Links http://cms.csr.nih.gov/publications/