1. The Ins and outs of nih peer review
Rebecca Wagenaar-Miller, PhD
Scientific Review Officer
National Institute of Dental and Craniofacial Research
NIH

2. Context-My Background
PhD Vanderbilt University Department of Cancer Biology (2002)
Matrix Metalloproteinase-7 in Mouse Models of Intestinal Tumorigenesis
Vanderbilt University Postdoctoral Fellow ( )
MMP-7 in pre-clinical model of colorectal cancer
NIH, NIDCR Postdoctoral Fellow ( )
Type II transmembrane serine proteases
MMP-mediated and uPARAP-mediated collagen degradation in development
Scientific Review Officer (SRO), NIDCR, NIH (2007-present)

3. Goals Give a brief look into the NIH peer review process
Update you on changes in NIH peer review
Discuss strategies for a successful application
Provide a few resources

4. Is this your view of the NIH?

5. How about this?

6. Or this?

7. Research grant Review process
National Institutes of Health
Research
Grant
Application
School or Other
Research Center
Center for Scientific Review
Assigns to IRG/Study Section & IC
Study Section
Initiates
Research
Idea
Submits Application
Evaluates for Scientific Merit
Institute
Evaluates for Program Relevance
Advisory Councils and Boards
Allocates Funds
Conducts
Research
Recommends Action
Institute Director
Takes final action for NIH Director

8. The peer review process

9. THE MISSION OF PEER REVIEW
To ensure that grant applications and contract proposals are evaluated by qualified scientific peers in a fair, objective, and timely manner.

10. The NIH peer review process
Conducted according to the Federal Advisory Committee Act
Meetings are closed to the public
All materials and discussions – strictly confidential
Over 80,000 applications reviewed per year
Almost 18,000 reviewers
GOALS

11. NIH Application review
80% of applications are reviewed by CSR in Scientific Review Groups (SRGs) that are part of Integrated Review Groups (IRGs)
The rest are reviewed by individual ICs
Requests for Applications (RFAs)
Program Announcements with IC Review (PARs)
Training Applications (F, K’s)
‘Complex’ Applications (P, U’s, Clinical Trials)
Contract Proposals

12. Summary statements The legal summary of the review meeting
Includes all three reviewer critiques, largely unedited
Includes a resume, which is a summary of the discussion of the meeting, focusing on the major strengths and weaknesses that resulted in the overall impact score
Used by council, applicants, program staff

13. grants and peer review life cycle
Award and Post Award
Pre-application
Peer Review
Council
Program
Scientific
Review
Officer
Program
and GM
Peer Review Meeting
Applications Arrive
Summary Statement
Council
2-3
months
2-6
weeks
One
month

14. The NIH Peer Review Process
To Request a Scientific Review Group
Cover letter of application
Application title
FOA# and title
Request:
Assignment to particular SRG or study section
Assignment to particular IC for funding consideration
Disciplines involved, if multidisciplinary
Explanation for late application
Not all requests
can be honored.
SRG rosters are posted 30 days before the SRG meeting:

15. The NIH Peer Review Process
Reviewer Assignments
For each application:
≥ Three qualified reviewers are assigned (“2 + 1”)
Assignments are made by the SRO
Based on the scientific content of application
Expertise of the reviewer
Suggestions from the PI on types of expertise –
not names!
Suggestions from Program staff
Assignments are
confidential!

16. 9 Point rating scale
Impact
Score
Descriptor
Additional Guidance on Strengths and Weaknesses
High Impact 1
Exceptional
Exceptionally strong with essentially no weaknesses 2
Outstanding
Extremely strong with only negligible weaknesses 3
Excellent
Very strong with only some minor weaknesses
Moderate Impact 4
Very Good
Strong but with numerous minor weaknesses 5
Good
Strong but with at least one moderate weakness 6
Satisfactory
Some strengths but also some moderate weaknesses
Low Impact 7
Fair
Some strengths but with at least one major weakness 8
Marginal
A few strengths and a few major weaknesses 9
Poor
Very few strengths and numerous major weaknesses
Non-numeric score options:  NR = Not Recommended for Further Consideration, DF = Deferred, AB = Abstention, CF = Conflict, NP = Not Present, ND=Not Discussed
Minor Weakness: An easily addressable weakness that does not substantially lessen impact
Moderate Weakness: A weakness that lessens impact
Major Weakness: A weakness that severely limits impact 1 3 2

17. The NIH Peer Review Process
NIH Scoring System
Preliminary scores (before the meeting)
Entered by assigned reviewers and discussants in
secure website
Made available to other SRG members
Final overall impact/priority scores (at the SRG meeting)
Voted by private ballot
All eligible members vote
Reviewers are instructed to revise their
criterion scores after the meeting.

18. The NIH Peer Review Process
Post-submission Materials
Applications submitted for Sept. 25th, 2010 and later:
Will only accept administrative materials resulting from unforeseen administrative issues, such as
Revised budget page(s) (e.g., due to new funding)
Biographical sketches (e.g., due to the loss of an investigator)
Letters of support or collaboration (e.g., due to the loss of an investigator)
Adjustment resulting from natural disaster (e.g., loss of animal colony)
Adjustments resulting from change of institution (e.g., PI moves to another university)
News of an article accepted for publication (do not sent article)
Special provisions for training grants and certain FOAs
Notice OD

19. The NIH Peer Review Process
SRG Meeting Procedures
Discussion format
Members with conflicts excused
Initial levels of enthusiasm stated(assigned reviewers and discussants)
Primary reviewer - explains project, strengths, weaknesses
Other assigned reviewers and discussants follow
Open discussion (full panel)
Levels of enthusiasm (assigned reviewers) restated
Individual SRG members vote
Other review considerations discussed (budget)

20. Peer review changes

21. New NIH Application Formats
NEW Sections
OLD Sections
INTRODUCTION-Revision (1 page)
INTRODUCTION-Resubmission (3 pages)
SPECIFIC AIMS (1 page)
RESEARCH STRATEGY (R01-12 PAGES)
R01 25 pages
Significance
Background and Significance
Innovation
Approach-
Research Design and Methods
-(preliminary studies-part of approach)
Preliminary Studies
-(progress report-part of approach)
Progress report

22. Scoring and review criteria
Research Project Grants
(R01, R21, R03)
Fellowship
(F30, F32, F32, F33)
Career Development
(K02, K08, K23, K25, K99)
OVERALL
Overall Impact
Overall Impact/Merit
Scored
Review
Criteria
Significance
Investigator(s)
Innovation
Approach
Environment
Fellowship Applicant
Sponsors, Collaborators, and Consultants
Research Training Plan
Training Potential
Institutional Environment & Commitment to Training
Candidate
Career Development Plan/Career Goals & Objectives/Plan to Provide Mentoring
Research Plan
Mentor(s), Co-Mentor(s), Consultant(s), Collaborator(s),
Environment & Institutional Commitment to the Candidate
Additional
Protections for Human Subjects
Inclusion of Women, Minorities, & Children
Vertebrate Animals
Biohazards
Resubmission
Renewal
Revision
Consider-
ations
• Applications from Foreign Organizations
• Select Agents
• Resource Sharing Plans
Budget & Period of Support
Training in the Responsible Conduct of Research
Applications from Foreign Organizations
Select Agents
Resource Sharing Plans
Impact Score
Do not Impact Score

23. Overall impact/priority score
Research Project Grant (RPG): likelihood for the project to exert a sustained, powerful influence on the research field(s) involved, in consideration of the five core review criteria, and additional review criteria
Fellowship (F): likelihood that the fellowship will enhance the candidate’s potential for, and commitment to, a productive independent scientific research career in a health-related field, in consideration of the scored and additional review criteria
Career Development (K): likelihood for the candidate to maintain a strong research program, taking into consideration the criteria in determining the overall impact/priority score

24. New Investigators and EARLY STAGE INVESTIGATORS

25. New investigator Background: Goals:
Concern about the number of New Investigators (NI) and the increasing age at first award
Goals:
Enrich the applicant pool with enough Early Stage Investigator (ESIs) to reduce the average age at first award
Accelerate the period of training leading to independence
Background: NIH has a long standing commitment to training and research funding of new investigators, understanding that they are the innovators of the future, bringing fresh ideas and technologies to existing biomedical research problems, and pioneering new areas of investigation.

26. Early Stage Investigators
NIH created a new ‘Early Stage Investigator’ (ESI) category designed to accelerate the early transition of new scientists to research independence by receiving their first R01 earlier. files/NOT-OD html
A PD/PI who qualifies as a New Investigator is considered an Early Stage Investigator (ESI) if he/she is within 10 years of completing his/her terminal research degree or is within 10 years of completing medical residency (or the equivalent). 

27. New investigator/early stage investigator
New Investigator Policies are limited to applications for traditional research project grant (R01) support.
All New Investigators are encouraged to apply for NIH R01 awards.
During Peer Review:
NI/ESI applications will be clustered during review
Consider the career stage of the PD/PI when evaluating elements of the applications, such as the availability of preliminary data
NOT-OD September 26, 2008

28. Implementation of ESI Definition
The NIH modified the collection of information on degree dates and medical residency within the personal profile of the eRA Commons.
PD/PIs must update their personal profile in the eRA Commons in order to be considered for the ESI classification.  Investigators who enter degree and residency completion dates will be notified of their ESI status by .
A procedure and guidelines for requesting an extension of the period of ESI eligibility is in place to accommodate individuals with various medical concerns, disability, pressing family care responsibilities, or active duty military service (instructions in Commons).

29. NI AND ESI POLICY FY 2010
Over the past three years, about 25 percent of all competing R01 awards have gone to New Investigators
Continue to equilibrate success rates for established and New Investigators submitting new R01 applications
Should permit the NIH to support 1,650 or more New Investigators
A majority of the New Investigators will be Early Stage Investigators
NOT-OD ,January 8, 2010

31. Tips for a successful grant application

32. Grant Writing for Success
Start early
Start with a good idea
Talk to your NIH Program Official(s)
Remember review criteria
Show you draft application to colleagues
who do not already know what you intend to do
who are not your best friend
who has/had the same type of grant
Follow instructions carefully 32 32

33. Foa SOURCE OF REVIEW GUIDELINES

34. Make YOUR strengths the focus of your application
Propose experiments that take advantage of
YOUR training
YOUR expertise
YOUR environment
Don't say that you're going to learn a sophisticated new technique and then apply it in later years
If you need a collaborator(s), get a strong one who publishes in peer-reviewed journals and who has a grant

35. Focus!
A major problem with applications from first time investigators is being overly ambitious.
Focus
Did I remember to say “Focus”?
Be certain every aim and experiment is clearly related to the overall goal of the application.

36. grants1.nih.gov/grants/guide/listserv.htm

37. Insider tips and tricks
Late submission policy for NIH study section service for any PI/PD of grant application (NOT-OD )
Standard receipt dates (1 or 2 weeks late)
“Late applications have been accepted for reasons such as…temporary or ad hoc service on a NIH extramural peer review group”
“No NIH staff member whether in the Center for Scientific Review or any of the other Institutes/Centers has the authority to give permission in advance for a late application.”
Cover letter important
Identify study section
Identify areas of expertise needed
Can list conflicts that should not review application (but be prepared to justify)
Submit to R01 to take advantage of NI/ESI mandate
Submit to RFAs if possible… free review of application

38. Research Portfolio Online Reporting Tool (RePORT) http://report. nih
A Searchable database of federally supported biomedical research
Access reports, data, analyses, expenditures, results of NIH supported research activities
Identify, Analyze IC(s) research portfolios, funding patterns, funded investigators:
Identify areas with many or few funded projects
Identify NIH-funded investigators and their research
Identify potential mentors/collaborators 38

39. Websites NIH Guide: http://grants.nih.gov/grants/guide/
NIH Office of Extramural Research
NIAID “how to” website for developing a grant application:
Writing a Grant: A Technical Checklist from NINDS:
 NIH RePorter: database of all funded grants including abstracts, funding amounts, study section, PI, and much more info:
NIH Peer Review Policies and Practices
Changes to Peer Review (and applications) peer-review.nih.gov/docs/application_changes.pdf

40. Thanks