Reviewing the Applications & Preparing for the Review School Turnaround AmeriCorps FY13 Peer Review Orientation Session IV

Objectives of this Training  Emphasize review expectations  Explain how to navigate through the Review  Discuss considerations for participating in the Panel Discussions  Describe the Individual Review Forms (IRFs) and how they are used  Identify Resources to help with High-Quality IRFs  Reference the Close Out process  Revisit Steps for Launching the Review  Ensure that you feel prepared to begin the review 2

Assessment Preview Complete and send to Subject: Orientation III 1) If your panel is flagged for Mid-Review Quality Control, what does that mean your panel needs to do? 2) Which role(s) review your IRFs after you have drafted it? 3) Name the three points when you would need to revise your IRF? (after____, after_____, and after_____) 4) What two things MUST align on your IRF? 5) Who will compile the Applicant Feedback Summary for each application? 3

Assessment Preview 5) Now that you have completed all of the Orientations, Please provide a question of what you feel is still unclear to you. In what area might we need to clarify, or provide additional guidance? 6) What will you need to receive in order to mark the launch of the review? 7) When are the Applicant Feedback Summaries created? 8) Identify two occasions when the Rubric should be used. 4

Review Preparation  Complete all training requirements  School Turnaround AmeriCorps FY13 Review Handbook  The Notice & Application Instructions  Orientation Sessions I-IV  Familiarize yourself with tools  Make schedule accommodations  Receive Panel Assignments  Download Assigned Applications  Participate in Panel Intro Call (order of applications is determined)  Review each application for COI  Begin Review! 5 Review Applications for COI Review Applications for COI Download Assigned Applications Participate in Orientation Activities

IRF Development Process 6 Complete the IRF DraftSubmit to Panel Coordinator (PC)Panel DiscussionAmend IRF PC (& Liaisons) Reviews Comments, Provides Feedback Incorporate FeedbackFinalize IRF

Review of Applications  Applications will generally be reviewed and discussed in groups  Read the full application (within page limit)  Applications should be evaluated against the Selection Criteria, not other applications  Apply your experience and expertise with balance  Complete your assessment of the Selection Criteria elements in your IRF  Select Ratings and comment on Strengths and Weaknesses  Utilize appropriate Forms as a resource  Review Rubric  Example IRF 7 Review Applications

Completing the IRF  Components of IRF  Heading Information  Instructions & Selection Criteria  Ratings, Total Score  Comments 8

Rubric Reference REVIEW RUBRIC  For each of the Selection Criteria, you will assign one of five possible Ratings  Excellent  Above Average  Average  Below Average  Poor  Each Rating has a CNCS-defined standard & corresponding numerical point value 9

Panel Discussions  There is a Panel Discussion for each application  Led by the Panel Coordinator  Estimate approximately 45 minutes per application  Assessments should not be shared with other Reviewers in advance  Purpose of the Panel Discussion  Open forum to discuss and re-consider your assessment of an application  Ensure fairness in the review  Ensure Selection Criteria is basis for assessment  Discuss points of agreement and disagreement  Mid-Review Quality Control  Serving as an Effective Panel Member  Be Prepared, on time  Responsive to Feedback 10

Mid-Review Quality Control 11 Revisit Panel Discussion Revisit Individual Reviewers Form (both Ratings and Comments) Submit revisions (to PC…PC submits to GARP Liaison)

Resources for Quality IRFs Writing Meaningful Comments & Example IRF  Application information is limited to the reviewed application and no others (no comparisons with other applications, etc.)  Only include comments that address School Turnaround AmeriCorps Selection Criteria  Reflect writing that is clear and concise  Ensure comments do not contradict each other  Ensure comments are aligned with and support the Rating for each section  Language is evaluative and does not restate information from Application  Appropriate comments (no suggestions for a “better proposal”)  Comments are relevant: Strengths and Weaknesses with greatest impact on the Rating  Avoid referencing Page Numbers  Grammar and spelling are correct (Spell AmeriCorps properly) 12

Expectations for Close-Out  Documents are labeled, verified, clean  Individual Reviewer Forms  Applicant Feedback Summaries  Panel Coordinator Notes  Confirm requirements are met with PC and GARP Liaison  Complete Evaluations  Confirmation received of approval for Honorarium 13

Expectations and Steps 14 Ensure your Participation is Verified Familiarity with Materials Receive Panel Assignment

Logistical Responsibilities  Use your Resources  Stay Connected  Accountability for your work  Submit COI and Participant Agreements  Banking Information for Honorarium (if applicable) 15

CNCS Staff Liaisons  GARP Liaison  Program Officer Liaison 16

Assessment for Orientation IV Complete and send to Subject: Orientation IV 1) If your panel is flagged for Mid-Review Quality Control, what does that mean your panel needs to do? 2) Which role(s) review your IRFs after you have drafted it? 3) Name the three points when you would need to revise your IRF? (after____, after_____, and after_____) 4) What two things MUST align on your IRF? 5) Who will compile the Applicant Feedback Summary for each application? 17

Assessment (cont.) 6) Now that you have completed all of the Orientations, Please provide a question of what you feel is still unclear to you. In what area might we need to clarify, or provide additional guidance? 7) What will you need to receive in order to mark the launch of the review? 8) When are the Applicant Feedback Summaries created? 9) Identify two occasions when the Rubric should be used. 18