1. Look Beneath the Surface Regional Anti-Trafficking Program
July 26, 2017

2. Overview of Reviewer Training
Reviewer Responsibilities
Writing Evaluative Comments
Scoring Applications
Panel Meetings
Application Review Module (ARM) Training

3. Reviewer Responsibilities Write Strengths and Weaknesses
Goal:
To write evaluative comments that are not judgmental but constructive.
What are “evaluative” comments?
Evaluative comments assess the value, worth, or quality of the information in the application
Why are evaluative comments important?
They assist ACF in the Federal funding process
They serve as technical assistance to applicants

4. Reviewer Responsibilities Write Strengths and Weaknesses
A comment is acceptable if it:
Draws from the evaluation criteria when articulating a basis for each conclusion
Uses examples from the application to support each finding
Provides the applicant with a sufficient rationale for the score
Example: “The applicant’s implementation strategies clearly outline the current collaborative and community relationships that have been established to support the proposed project. Letters of Support and a signed MOU details the role and specific contributions of each partner.” p. 15, and Appendix A.” (strength)

5. Reviewer Responsibilities Write Strengths and Weaknesses
A comment is unacceptable if it:
Reflects bold, insulting opinions – “The panel saw no reason to consider funding this proposal.” (weakness)
Reflects erroneous statements or opinions – “There is no way this organization can do this project.” (weakness)
Does not concisely detail the basis for a conclusion – “The applicant identifies results and benefits.” (strength)

6. Reviewer Responsibilities Write Strengths and Weaknesses
Sample Evaluative Comments – Strength
Not Acceptable:
A timeline for the project is provided.
Better:
The applicant has included a realistic timeline and has provided a chart showing the proposed timeline (p.9).
Best:
The applicant provided a realistic timeline and a comprehensive work flow model identifying the timeframe for performing and/or completing key project activities. The work flow model captured the role of each partner and the anticipated project milestone. (pp. 9-11).

7. Reviewer Responsibilities Write Strengths and Weaknesses
The Chairpersons and PAMs will reject comments that:
state that the applicant has met the guidelines without providing the basis for how this conclusion
do not provide adequate detail and does not cite application page numbers
are poorly written or unintelligible
do not adequately justify scores
are duplicates or if two comments conflict
are mixed (strength and weakness in same comment)
do nothing but restate announcement
are offensive or not based on criteria in the announcement

8. Reviewer Responsibilities Scoring Applications
Requirements for Scoring:
Ensure that the scores are consistent with the written comments
Use the full range of points, if appropriate
Score applications independently
Verify all scores
Don’t try to produce one or more “winning” applications

9. Reviewer Responsibilities Scoring Applications
Avoid:
Giving a perfect score to a criterion or element that has even one stated weakness
Listing weaknesses that are not part of the evaluation criteria
Providing an applicant with a high score with insufficient comments to justify it OR providing an applicant with a low score with insufficient comments to justify it.

10. Reviewer Responsibilities Scoring Applications
If you think something is questionable, please bring it to the attention of your Chairperson.

11. Reviewer Responsibilities Participate in Panel Meetings
Contribute to panel discussions of each application’s strengths and weaknesses
Assist the panel Chairperson in completing a panel summary report for each application
Assist the panel Chairperson with revisions to the panel summary report

12. Reviewer Responsibilities Participate in Panel Meetings
Performance Expectations:
Be prompt and fully prepared for all scheduled conference calls
Achieve full participation of all members
Maintain objectivity and make thorough, comprehensive analysis of applications
Demonstrate respect for fellow panel members – respect their persons and respect their opinions

13. Reviewer Responsibilities Participate in Panel Meetings
Performance Expectations, cont.:
Practice problem solving and team building
Develop the panel summary report as a team
Maintain confidentiality
The Chairperson should feel free to ask for assistance when needed

14. Reviewer Responsibilities Submit Comments to Chairperson
Submission process:
Enter comments into ARM system online within requested timeframe
Revise comments as requested by Chairperson
Discuss comments with panel
Revise comments/scores as needed per panel discussions
Revise comments as needed per Chairperson/PAM requests

15. ARM and Grant Review Technical Assistance
ARM Technical Assistance
By Phone: M-F 8:30 AM – 5:00 PM (ET) By
Grant Review Assistance
Stephanie McDaniel:
Brett Heiney:

16. THANK YOU !!