Preparing for NIH Peer Review Thomas M. Vollberg, Ph.D. Chief, Scientific Review Branch vollbert@mail.nih.gov

Types of NIH Funding Opportunities Grants - NIH Guide – Funding Opportunity Announcements Program Announcements (PA) Parent PAs , e.g. parent R01, parent R03… Vehicle for submitting investigator-initiated research Institute-Specific PAs - stated area of scientific interest Institute Specific PA with special referral/review characteristics (PARs) Requests for Applications (RFAs) Institute-targeted research area with designated funding Contracts – NIH Guide and FedBizOpps Requests for Proposals (RFPs) Solicitation for research and development contract proposals NIH-defined scope of work and research activity

Review Cycles – Grant Applications Receipt Review Council Jan-April June-July October May-Aug Oct-Nov January Sep-Dec Feb-Mar June Almost all applications are now submitted electronically either via Grants.gov or ASSIST.

Receipt and Referral: Assignment of Applications for Programmatic Consideration and Review All applications are received by the Center for Scientific Review (CSR) where the Division of Receipt and Referral uses established guidelines to make two assignments: to an Institute or Center (IC) for program consideration and to either CSR or an individual IC for determination of scientific merit by a peer review panel http://grants.nih.gov/grants/how-to-apply-application-guide/forms-d/general/g.600-phs-assignment-request-form.htm

Review Process: CSR or IC as Locus of Review? Peer review assignment to CSR or an I/C is based on two factors: The “kind”of application as defined by the mechanism Institute-specific initiatives (Funding Opportunity Announcements) CSR typically reviews 60% of all applications and the various I/Cs review the other 40%

Study Section versus Special Emphasis Panel Study Sections - at CSR Clustered into Integrated Review Groups by general science area Applications assigned by science area to IRG and to a Study Section Standing Study Section Membership Permanent members, appointed to serve, and Ad Hoc members, invited for a meeting usually to fill an expertise area. “Applications will be evaluated … by (an) appropriate Scientific Review Group(s), in accordance with NIH peer review policy and procedures, using the stated review criteria. “ Special Emphasis Panel – at I/C or at CSR [see FOA] Ad Hoc Membership, recruited to review a specific group of applications in response to a PAR or RFA. “Applications will be evaluated ….by (an) appropriate Scientific Review Group(s), convened by [I/C or CSR], in accordance with NIH peer review policy and procedures, using the stated review criteria.” PHS Assignment Request Form http://grants.nih.gov/grants/how-to-apply-application-guide/forms-d/general/g.600-phs-assignment-request-form.htm

Review Process: Mechanisms Reviewed in CSR Investigator-initiated: Research Project Grants (R01) Exploratory and Small Grants (R21, R03) AREA Grants (R15) Small Business Grants (SBIRs, STTRs) Fellowship Grants (F32s, F31s) FOAs: Program Announcements (PAs, PARs, PAS). 20,000 applications per review cycle Ks -- Twelve Different Awards Vary across Institutions – talk to appropriate Program Official! Articulation with Career Stage Interaction with other NIH Awards R03 – limited to two years/$50,000 per year; little preliminary data required R21 – vary by Institute, generally exploratory and high-risk; limited to 2 years with total direct costs of $275,000 AREA – eligible Institutions as defined by total NIH support; PI requirements as defined by other support; small research projects with $150,000 direct costs over 3 years; undergraduate component. R01s – modular equal to or below $250,000 direct costs per year; greater than $250,000 must provide detailed budgets.

CSR www.csr.nih.gov

Review Process: Reviewed in NIMHD Scientific Review Branch Institute-Specific FOAs (RFAs, PARs): RFA Initiatives for Research Project Grants (R01) Cooperative Agreement (U01) Loan Repayment Program (LRP) Conference Grants (R13s) Specialized Center-Cooperative Agreements (U54) Resource-Related Research Projects-Cooperative Agreements (U24) Resource-Related Research Projects (R24) Technology Transfer and Small Business special programs (R41-R44) Pathway to Independence (K99/R00) and Research Training (T32) ~300 applications per review cycle ZMD1

First Level of Peer Review: Meeting - Review Criteria Overall Impact Core Review Criteria Significance, Investigators, Innovation, Approach, Environment Additional Review Criteria (No individual scores, but factor into an overall Impact Score) Protection for human and/or vertebrate animals, inclusion of WMC, resubmissions, renewals or revision Additional Review Considerations (Do not receive any score and do not influence the overall Impact Score) Foreign organizations, select agents, resource sharing plan, budget

First Level of Peer Review: Meeting - Scoring IMPACT is the likelihood of an application to exert a sustained, powerful influence on the research field(s) involved, in consideration of all review criteria as specified in the FOA. The assessment of IMPACT is given as an integer with a score range from 1 (best possible score) to 9 (worst possible score). All reviewers (not in conflict) at the meeting assign a score for each application that is discussed (usually the “upper half”). Applications that are determined by the panel to be non-competitive (generally the lower half) receive a rating of “Not Discussed”. The goal of the discussion is to reach an understanding of the IMPACT for an application. Impact scoring guidance to reviewers sets scores from 1-3 as high impact, 4-6 as moderate impact and 7-9 as low impact and gradations from lower to higher integers to cover an increasing accumulation of minor to major weakness. Preliminary individual ratings of assigned reviewers are provided ahead of the meeting to guide consideration of applications for discussion.

First Level of Peer Review: Impact Score Discussed applications are voted an impact score (1-9) from all eligible members of the panel individually. The two-digit, IMPACT SCORE which appears on the face page of a Summary Statement is calculated by averaging the impact integer scores from all eligible reviewers and multiplying by 10. In some cases, IMPACT SCORES within a study section are normalized across several rounds of review and converted to percentiles. Percentiles – apply to panels that meet regularly. Special Emphasis Panels – meet once, ad hoc, and no percentiles are applied.

First Level of Peer Review: Summary Statement The Summary Statement constitutes the official report of the outcome of review for an application to inform I/C Council(s) and NIH Program Staff. As a secondary purpose, the summary statement serves to communicate the review outcome to the PDs/PIs. A completed Summary Statement is made available through the NIH eRA database for privileged access by I/C Council members, NIH program staff, grants management staff and the applicant. On Face Page – Panel assessment of IMPACT SCORE – and a percentile if applicable Also, Panel assessment for HS, Inclusion Plans and Vertebrate Animals Other project information. Title, PI Name, Institution, Budget requested. Study Section – SRO identifier Program Officer assignment

Structure of FOAs Part 1. Overview – Executive Summary Sponsoring funding sources, grant mechanism, purpose, dates Part 2. Full Text Section I. Full description of the scientific purpose Section II. Award Information: application types (new, renewal, resubmission, revision), funding, budget limitations, award period Section III. Eligibility, required registrations. Section IV. Application and Submission Information Getting an application package, content and form of application, page limitations, are letters of intent requested?, etc. Section V. Application Review Information 1. Criteria – review criteria and review considerations 2. Review and Selection Process – Review panel and program assignments Section VI. Award Administration

Resources for Grants Process and Peer Review Process NIH Office of Extramural Research Resources for Researchers Finding funding opportunities, grants process, applying, NIH electronic system for administering applications and grants, etc. http://grants.nih.gov/grants/information-for-research.htm Access to CSR information on Planning, Writing and Submitting Applications, CSR Receipt and Referral and Initial Review, Results and Appeals http://public.csr.nih.gov/ApplicantResources/Pages/default.aspx

Summary of Helpful NIH Web Pages Office of Extramural Research (OER) Web Page: http://grants.nih.gov/grants/oer.htm NIH Grants Policy Statement (Rev. 11/16): http://grants.nih.gov/grants/policy/nihgps/ NIH Extramural Nexus – newsletter for the extramural community: http://nexus.od.nih.gov/all/nexus-by-date/ Grant Application Basics: http://grants.nih.gov/grants/grant_basics.htm eRA Training: Video Tutorials http://era.nih.gov/era_training/era_videos.cfm

Summary of Helpful NIH Web Pages Applying Electronically: http://grants.nih.gov/grants/ElectronicReceipt/index.htm Ten Checks to Help Avoid Common Application Errors: http://grants.nih.gov/grants/how-to-apply-application-guide/learn-how-we-check-your-application-for-completeness/avoiding-common-errors.htm Do I have the right electronic forms for my NIH application?: http://grants.nih.gov/grants/ElectronicReceipt/files/right_forms.pdf Self Help Resources page: http://grants.nih.gov/support/index.html

Summary of Helpful NIH Web Pages eRA Commons Web pages: http://era.nih.gov/ eRA Commons User Guides: http://era.nih.gov/commons/user_guide.cfm Intellectual Property Policy: http://grants.nih.gov/grants/intell-property.htm Research Portfolio Online Reporting Tools (RePORT): https://report.nih.gov RePORT Expenditures & Results (RePORTER): https://projectreporter.nih.gov/reporter.cfm Availability of Resources for Instruction in the Responsible Conduct of Research http://grants.nih.gov/grants/guide/notice-files/NOT-OD-16-122.html

NIH OER Listservs NIH Guide for Grants and Contracts: Official publication for NIH Grant Policies, Guidelines & Funding Opportunities http://grants.nih.gov/grants/guide/listserv.htm Office for Human Research Protections (OHRP): http://www.hhs.gov/ohrp Office of Laboratory Animal Welfare (OLAW): http://grants.nih.gov/grants/olaw/references/list.htm eSubmission: Separate listservs available for scientists and administrators http://grants.nih.gov/grants/ElectronicReceipt/listserv.htm

Grants Information: Who to Contact? General Application Questions: E-Mail: GrantsInfo@nih.gov Phone: 301-435-0714 Grants.gov Customer Support: E-Mail: support@grants.gov Webpage: http://grants.gov/ Phone: 1-800-518-4726 eRA Commons Helpdesk: Web: http://era.nih.gov/help/ Toll-free: 1-866-504-9552 Phone: 301-402-7469 Hours: Mon-Fri, 7a.m. to 8 p.m. Eastern Time