1. Health Resources and Services Administration Division of Independent Review HRSA Maternal and Child Health Bureau State Systems Development Initiative Grant Program – Tier 2 Field Pre-Review Orientation
Vicky Johnson
Lead Review Administrator
September 22, 2017

2. DIR Mission
To ensure a fair, ethical and objective review of each application

3. Conflict of Interest
A conflict of interest occurs when a reviewer, close relative, or professional associate of the reviewer has a financial or other interest in an application that is likely to bias the reviewer’s evaluation of that application.

4. Confidentiality Before, during, and after the review.
Results of all conference calls are CONFIDENTIAL.
Discussions regarding applications should be confined to any conference calls.
All review materials are to be destroyed after the meeting has concluded.

5. Approach Avoid personal bias.
Independently review, evaluate and score each application based on the Review Criteria.
Consistently evaluate each application (i.e., scores should reflect the strengths and weaknesses noted).
Review Applications in Assigned Order!

6. Outside Information
If information is not in the application the reviewer should review as though the information does not exist.
Applications should be reviewed and evaluated in accordance with the Review Criteria .
Applications should not be compared.
DO NOT discuss or mention any information that is not contained in the application itself (including web- site references).

7. Role of the Participants
DIR Review Administrator (Vicky L. Johnson) (RA): Guides, oversees, and verifies the integrity of the objective review process.
Program (Scott Snyder): When requested, responds to specific technical and programmatic issues.
Grants Management (Ernest Charles): When requested assists with budget issues.
DIR Contractor (Humanitas – Alex Swetlow): Provides logistical support to DIR such as honoraria and reimbursements, transmission of review materials and any other special needs.

8. Steps in Conducting a Field Review
1.Evaluation of Assigned Applications: Read the NOFO before reading the assigned applications. Your reviewer checklist will match the review criteria to which applicants respond.
The Checklist is for you to provide your evaluation of the application. Check either the strength, met, or weakness box and provide your clear, organized statements of strengths and weaknesses.
The Checklist with your comments are due back to Alison at Humanitas by 1st & 2nd by 09/28 and 3rd & 4th by 10/5 with last one due 10/12. Two checklists per week.

9. Steps in Conducting a Field Review cont.
2. Criterion Scoring: Use the Scoring Rubric along with the Score Worksheet to compile your scores and budget information.
All scores and budget amounts are to be entered into ARM. (Use whole numbers and not fractions).
All completed Summary Statement Checklists are to be sent to Alex no later than October 12, 2017.
The Scoring Rubric is provided to assist you in your scoring, there are 5 categories that are helpful and easy to follow.
The Score Worksheet is a courtesy document for you to use. It is not to be returned. All scores and budget numbers should be entered into ARM.
When you have finished entering scores and budget information into ARM, you must click the “submit to chair” button.
The scoring rubric

10. Sample Reviewer Score Worksheet
For Your Records

11. Steps in Conducting a Field Review cont.
3. Criterion Scoring and Comments:
Please remember that comments are NOT to be entered into ARM. You should only be entering the following into ARM:
scores
budget numbers

12. ARM: Enter Scores and Budget Data ONLY!

13. Steps in Conducting a Field Review cont.
4. Budget:
Reviewers are to provide budget recommendations for 5 years.
Budget numbers are to be entered into ARM.
Reviewers cannot recommend an increase in the Applicant’s budget request. If a reduction in the budget request is recommended, Reviewers must provide a rationale.

14. Summary Statement Development
Please provide clear and useful feedback in the Corresponding Strength and Weakness comments.
We are especially interested in the value of the comments, particularly in areas of weaknesses.
Each statement must be properly justified. If your check the strength or weakness box, there must be a corresponding strength and or weakness provided in the designated section.

15. Summary Statement Development
A strength is defined as a response that clearly meets or substantially exceeds requirements set forth in a review criterion.
A weakness is defined as a response that falls short of meeting requirements set forth in a review criterion.
Please use complete present tense sentences.

16. Logistics
All Reviewers should start with the first application on their assignment sheets.
Reviewers should turn in their worksheets as soon as they are done with each of their assigned applications
Please Note: If there is a discrepancy in score between the three primary reviewers, we will schedule a “Call Back”, to have them discuss the issues with the application.
Call Back can last from 20mins to 1 hour.
Call Backs are currently scheduled to occur between Oct 18-20,

17. Deletion of Files
At the end of this review, HRSA expects that you will permanently delete all review related files from your electronic devices.

18. Questions During the Review Period
If you have questions regarding the NOFO and or Budget, Please me at:
I will ensure that your questions are sent to the appropriate HRSA Staff.
If you are unsure of anything, please me with your questions and I will respond as quickly as possible!

19. SSDI Tier 2 Funding Opportunity HRSA-18-062 Reviewers Pre-Review Call
Scott A Snyder, MPH
Public Health Analyst
Division of State and Community Health (DSCH)
Maternal and Child Health Bureau (MCHB)
Health Resources and Services Administration (HRSA)

20. SSDI Purpose
Develop, enhance, and expand state and jurisdictional Title V MCH data capacity for its needs assessment and performance measure reporting in the Title V MCH Block Grant Program.

21. SSDI FY18 Competing Continuation
FY18 Notice of Funding Opportunity (NOFO) consists of 2 separate funding tiers, based upon the applicant’s access to national data for their Title V MCH Block Grant Program’s National Performance Measures (NPMs) HRSA Eligibility is limited to the 8 Jurisdictional Title V MCH Block Grant Agencies of American Samoa, Commonwealth of the Northern Mariana Islands, Federated States of Micronesia, Guam, Marshall Islands, Puerto Rico, Palau, and the U.S. Virgin Islands. HRSA States and the District of Columbia (DC)

22. Three Goals for Tier 2
Tier 2 applications for SSDI grant funding must address three goals - two general and one state-specific.

23. Goals for Tier 2 - General
Goal 1 (General) – Build and expand state MCH data capacity to support the Title V MCH Block Grant program activities and contribute to data-driven decision making in MCH programs, including assessment, planning, implementation, and evaluation. Goal 2 (General) – Advance the development and utilization of linked information systems between key MCH datasets in the state.

24. Goals for Tier 2 – State Specific
Goal 3 (State Specific) – Select one of the following special projects for programmatic focus over the 5-year funding period:
Support program evaluation activities around the NPMs that contribute to building the evidence base for the Title V MCH Block Grant;
Provide data support to states participating in quality improvement activities (e.g., Collaborative Improvement and Innovation Networks); or
Support surveillance systems development to address data needs related to emerging MCH issues (e.g., the Zika virus, neonatal abstinence syndrome (NAS), or lead poisoning prevention).

25. Activities for Tier 2 – General Goals
Supporting Title V data needs associated with their 5-year Needs Assessment process, including selection of state priorities, as well as ongoing interim needs assessment activities;
Assisting Title V with development, selection, refinement, and tracking of data and performance measures that are associated with their performance measure framework, including NOMs, NPMs, SPMs, and ESMs;
Supporting data needs associated with annual preparation of their Title V application/annual report; and
Developing and implementing a plan for overcoming barriers to data linkage across the 5-year funding cycle, particularly focusing on indicators from the Minimum/Core (M/C) Dataset for Title V MCH Block Grant programs.

26. Activities for Tier 2 – State Specific Goal
Tier 2 activities associated with the third state-specific goal are based upon the area selected for targeted emphasis. Activities identified will be unique to each SSDI grantee.

27. Tier 2 Program Outcomes Tier 2 program outcomes include:
Maintaining timely submission of data-driven 5-Year Needs Assessment;
Maintaining ongoing annual assessment updates through effective use of data;
Increasing states’ ability to select and track NPMs, SPMs, and ESMs that align with their Title V MCH Block Grant priorities;
Maintaining timely submission of Title V MCH Block Grant application/annual reports;

28. Tier 2 Program Outcomes (cont.)
Developing and implementing a plan for overcoming barriers to data linkage across the 5-year funding cycle, particularly focusing on indicators from the M/C Dataset for Title V MCH Block Grant programs;
Maintaining use of M/C Indicators to inform data-driven decision making;
Increasing linkages of MCH datasets to inform MCH program planning, evaluation, and improvement;
Increasing consistent, direct access to standard datasets that inform MCH program planning, evaluation, and improvement;

29. Tier 2 Program Outcomes (cont.)
Conducting activities associated with the state-specific goal based upon the area for targeted emphasis; and
Developing one or more products (e.g., data books, data dashboards, fact sheets, infographics, journal articles, tool kits, websites, and white papers) that demonstrate the use of data to inform decision making for improved MCH outcomes.

30. New SSDI Performance Measure
New for this 5 Year Project Period - Tier 2 applicants will be responsible to report on a new SSDI Performance Measure (CB 7), “The percent of programs promoting and facilitating state capacity for direct annual access to MCH electronic health data.” Goal - To ensure state capacity for accessing electronic health data on a timely basis for programming and/or reporting. Measure - % of programs that are consistently accessing direct electronic MCH health data to support planning, monitoring, and evaluation on a timely basis.

31. New SSDI Performance Measure
The new performance measure is not part of the required application. Applicants are not expected to submit a completed SSDI PM – CB7.

32. Project Narrative Introduction Needs Assessment
Methodology, Work Plan, and Resolution of Challenges
Evaluation and Technical Support Capacity
Organizational Information
Budget and Budget Narrative

33. Tier 2 – Review Criteria
Criterion 1: NEED (10 points) – Corresponds to Section IV’s Introduction and Needs Assessment
This criterion is related to the extent to which the applicant demonstrated a clear and comprehensive understanding of the conceptual issues important to the SSDI program. Specifically, this criterion assesses the manner in which the applicant described its understanding of the Tier-specific SSDI Goals.

34. Tier 2 – Review Criteria (cont.)
Criterion 2: RESPONSE (35 points) – Corresponds to Section IV’s Methodology, Work Plan, and Resolution of Challenges
This criterion is related to the extent to which the applicant’s proposed plan adequately addresses each of the identified Tier 2 SSDI program goals.

35. Tier 2 – Review Criteria (cont.)
Criterion 3: EVALUATIVE MEASURES (35 points) – Corresponds to Section IV’s Evaluation and Technical Support Capacity
This criterion is related to the extent to which the applicant outlines its methods to monitor and evaluate the project results.

36. Tier 2 – Review Criteria (cont.)
Criterion 4: IMPACT (10 points) – Corresponds to Section IV’s Methodology, Work Plan, and Resolution of Challenges
This criterion is related to the feasibility and effectiveness of plans for dissemination of project results, and the extent to which project results may be national in scope, and the degree to which the project activities are replicable, and the sustainability of the program beyond the federal funding.

37. Tier 2 – Review Criteria (cont.)
Criterion 5: RESOURCES/CAPABILITIES (5 points) – Corresponds to Section IV’s Methodology, Work Plan, and Resolution of Challenges, Evaluation and Technical Support Capacity, and Organizational Information
This criterion is related to the extent to which Tier 2 applicants demonstrate their ability to perform the proposed scope of work.

38. Tier 2 – Review Criteria (cont.)
Criterion 6: SUPPORT REQUESTED (5 points) – Corresponds to Section IV’s Methodology, Work Plan, and Resolution of Challenges and Section IV, 2, iii and iv’s Budget and Budget Narrative
This criterion is related to the extent to which costs, as outlined in the budget and required resources sections, are reasonable and support the scope of work; and the extent to which key personnel have adequate time devoted to the project to achieve project objectives.

39. HRSA DIVISION OF GRANTS MANAGEMENT OPERATIONS
GRANTS MANAGEMENT GUIDELINES TO HRSA REVIEWERS

40. GRANTS MANAGEMENT ROLE
Provide and interpret grants management policy to reviewers. Determine the allowability of budget items. Assist the program office in determining eligibility of an applicant.

41. Reviewer Role
Your role as a reviewer in the budget review is to use your expertise in determining the reasonableness of the proposed budget from the applicants.

42. EXPECTATIONS FOR HRSA REVIEWERS
Review the entire application, including the budget.
Determine whether proposed budgets are reasonable and necessary.
Determine if the time allocated by key personnel is realistic and appropriate to achieve project objective
Determine if budget line items are well described and justified in the budget justification.
Panel advice should be thorough.

43. BUDGET REVIEW
Completed Research & Related Budget Form and budget narrative has been submitted.
Detail cost breakout of direct costs provided.
Budget cost items are clear, concise and necessary for the proposed program activities.

44. BUDGET - Caps on expenses:
Many reviewers are aware of the Consolidated Appropriations Act 2016, enacted December 18, 2015 limits the salary amount that may be awarded and charged to HRSA grants and cooperative agreements. This act outlines salary limitations where HRSA funds cannot be used to pay the salary of an individual at the rate in excess of $187,000 for all grants awarded after January 1, 2017.
(This salary limit does not apply to affordable care grants)

45. Unallowable or Restricted Costs
International travel
Entertainment costs
Fundraising expenses
Lobbying expenses

46. Allowable Costs
Indirect Costs
Audit costs
Tuition Remission

47. Allowable Costs (cont.)
Incentive payments to volunteers or participants under study for the MCHB Research Program
Dues or membership fees
Bonuses/incentive payments for employees of the organization as part of a compensation package

48. BUDGET RECOMMENDATIONS
Approved as requested
Written recommendation for budget reduction

49. Budget Recommendation (cont.)
Recommendation of increase in budget is not allowed.
Please be aware the base salary or salary rate may not be recommended for reduction. The only exception to this if the base salary exceeds the salary cap of $187,000 as stipulated in the FOA for executive positions.

50. FINAL WORDS
Lastly…. Please be aware, Maternal & Child Health Bureau (MCHB) Program Office and Grants Management will decide how much the awardee will receive in funding using your scores and if applicable any written comments for reduction.
Staff will review budgets of approved applications in accordance with grants policies and procedures and take under advisement any committee recommended disallowed costs.

51. On behalf of HRSA, Thank You again for serving on this review committee!!