325K: COMBINED PRIORITY FOR PERSONNEL PREPARATION Webinar on the Annual Performance Report for Continuation Funding Office of Special Education Programs U.S. Department of Education

ANNUAL GRANT PERFORMANCE REPORT Provides an annual report of your performance on (a) meeting the approved objectives of the project, and (b) financial management of federal funds Required for all active grants, including those in no cost extension (NCE) Used by OSEP to determine if substantial progress has been made in order to grant continued funding or a NCE 2

Develop relevant, measurable, outcome-oriented performance measures that maximize the potential for meaningful data Complete the ED Grant Performance Report (ED 524B) Report OVERVIEW 3 Recognize strong project objectives that can be associated with high- quality performance measures Project Objectives Performance Measures

WHY IS THIS IMPORTANT? 4 Make it easier for you to measure your progress Establish targets (both short-term/annual and long-term) Allow you to report progress easily and quantitatively HIGH-QUALITY OBJECTIVES & MEASURES: Allow OSEP staff to gather evidence of program effectiveness Taken from the Center for Evaluation and Education Policy (CEEP) presentation:

GOAL - OBJECTIVES - MEASURES 5 Program Goals: What are the intend outcomes of the OSEP’s Personnel Development Program (Found in Priority) Project Objectives: What your project is doing to support the overall program goal (Found in your application) Performance Measures: How you measure your progress toward meeting your objectives (Program/GPRA, Project) Taken from the Center for Evaluation and Education Policy (CEEP) presentation:

WHAT ARE YOU TRYING TO ACCOMPLISH? PROJECT OBJECTIVES 6 Preferred format for objectives: Begin the objective with a verb and define a desired outcome or condition

HIGH QUALITY PROJECT OBJECTIVES 7 How relevant is the project objective to the overall goal of the program and/or the goal of your project? How focused is the project objective? How applicable is the project objective to the specific activities that are being conducted through your particular project? Are there concepts in the project objective that lend themselves to measurement? If so, is measurement feasible? RELEVANCEAPPLICABILITY FOCUSMEASURABILITY Taken from the Center for Evaluation and Education Policy (CEEP) presentation:

PROJECT OBJECTIVES: EXAMPLES 8 Recruit and retain high-quality scholars Implement a high-quality personnel preparation program that will promote competencies in using evidence-based practices Prepare high-quality personnel with the knowledge and skills to use evidence-based instruction and intervention practices to improve the outcomes of children with disabilities

PERFORMANCE MEASURES 9 How are you measuring your progress in meeting your objectives?

PERFORMANCE MEASURES ARE: 10 How will progress be assessed? Measurable indicators used to determine how well objectives are being met. How much progress will constitute success? How will it be known if an objective or part of an objective has been achieved? Taken from the Center for Evaluation and Education Policy (CEEP) presentation:

Project Objective 1 PERFORMANCE MEASURES 11 Performance Measure 1a Performance Measure 1b Performance Measure 1c Taken from the Center for Evaluation and Education Policy (CEEP) presentation:

Measures established by OSEP for program use. OSEP uses the GPRA measures as the Program measures required of all grantees. Each grantee reports on the approved project performance measures established to meet their project objectives. Project TYPES OF PERFORMANCE MEASURES 12 Measures established by OSEP and approved by OMB for the PDP for reporting to Congress under the Government Performance and Results Act of GPRA Program

PERSONNEL DEVELOPMENT PROGRAM PERFORMANCE MEASURES The percentage of Special Education Personnel Development projects that incorporate evidence-based practices into their curricula. The percentage of Special Education Personnel Development funded degree/certification recipients who are working in the area(s) for which they were prepared upon program completion, and who are fully qualified under IDEA. The percentage of scholars completing Special Education Personnel Development funded programs who are knowledgeable and skilled in evidence-based practices for infants, toddlers, children, and youth with disabilities. The percentage of Special Education Personnel Development funded scholars who exit preparation programs prior to completion due to poor academic performance. The percentage of Special Education Personnel Development funded degree/certification program recipients who are working in the area(s) in which they were prepared upon program completion. (Discontinued reporting): The percentage of Special Education Personnel Development degree/certification recipients who maintain employment in the area(s) for which they were prepared for 3 or more years, and who are fully qualified under IDEA. The Federal cost per fully qualified degree/certification recipient.

PILOT PERSONNEL DEVELOPMENT PROGRAM PERFORMANCE MEASURES The number and percentage of degree/certification recipients who are employed in high-need districts; The number and percentage of degree/certification recipients who are employed in the field of special education for at least two years; and The number and percentage of degree/certification recipients who are rated as effective by their employers. 14 Beginning in 2015, the Department also is gathering information on the following outcome measures:

PROGRAM PERFORMANCE MEASURES Grantees should propose a project performance measure that addresses each program performance measure reported for the 325K program. The focus of the project measure must be the same as and clearly aligned with the program measure. 15

PROGRAM PERFORMANCE MEASURE Example of related project performance measure: By the end of Year 1, all core program course syllabi will achieve a score of 90 or higher on its integration of evidence-based practice as measured by an expert panel using a rubric designed by the project. 16 #1. The percentage of Special Education Personnel Development projects that incorporate evidence-based practices into their curricula. (Process Measure)

PROGRAM PERFORMANCE MEASURE (1) Example of related project performance measure: By year 5 of the project, 95% of scholars will demonstrate knowledge and skills in all assessment competencies at an “acceptable” level or higher using a rubric designed by the project. 17 #2. The percentage of scholars completing Special Education Personnel Development funded programs who are knowledgeable and skilled in evidence-based practices for infants, toddlers, children, and youth with disabilities. (Outcome Measure)

PROGRAM PERFORMANCE MEASURE (2) Example of related project performance measure: As measured annually, no scholars (0%) will have exited the program due to poor academic or field-based performances. 18 #3. The percentage of Special Education Personnel Development funded scholars who exit preparation programs prior to completion due to poor academic performance. (Process Measure)

PROGRAM PERFORMANCE MEASURE (3) Example of related project performance measure: By the end of Year 5 of the project, 90% of scholars will be working in the area for which they were trained following program completion. 19 #4. The percentage of Special Education Personnel Development funded degree/certification program recipients who are working in the area(s) in which they were prepared upon program completion. (Intermediate Outcome Measure)

PROGRAM PERFORMANCE MEASURE (4) Example of related project performance measure: By the end of Year 5 of the project, 90% of program graduates will be working in the area for which they were trained and fully qualified under IDEA. 20 #5. The percentage of Special Education Personnel Development funded degree/certification recipients who are working in the area(s) for which they were prepared upon program completion, and who are fully qualified under IDEA.

PROGRAM PERFORMANCE MEASURE (5) Example of related project performance measure: By the end of Year 5 of the project, 90% of eligible program graduates will have maintained employment in the area for which they were trained for 3 or more years, and be fully qualified under IDEA. 21 #6. The percentage of Special Education Personnel Development degree/certification recipients who maintain employment in the area(s) for which they were prepared for 3 or more years, and who are fully qualified under IDEA.

PROGRAM PERFORMANCE MEASURE (6) Example of related project performance measure: The Federal cost is measured in Year 5 by dividing the total grant award (e.g., $250,000 x 5) by the number of scholars counted for Performance Measure #2. 22 #7. The Federal cost per fully qualified degree/certification recipient. (Efficiency Measure)

Measures established by OSEP for program use. OSEP uses the GPRA measures as the Program measures required of all grantees. Each grantee reports on the approved project performance measures established to meet their project objectives. Project TYPES OF PERFORMANCE MEASURES (1) 23 Measures established by OSEP and approved by OMB for the PDP for reporting to Congress under the Government Performance and Results Act of GPRA Program

PROGRAM PERFORMANCE MEASURE (7) Measures that the grantee establishes to meet their project objectives Project performance measures can address both the process of working towards an objective and the outcome related to meeting the objective Ensure a mix of both process and outcome measures 24 Project

PROGRAM PERFORMANCE MEASURE (8) What will change. How much change you expect. Who will achieve the change. When the change will take place. 25 High-quality performance measures show:

PROJECT PERFORMANCE MEASURE EXAMPLES 26 Process Measure Project staff (who) will implement 4 (how much) targeted recruitment activities (what ) during the first year of the grant (when). Outcome measure At the end of Year 2 (when), inquiries about the program (what) from potential candidates from underrepresented groups (who) will increase 20% (how much) from previous recruitment efforts during the first year of the grant. Outcome measure At the end of each semester (when), 100% of scholars (who) will rate their mentoring support (what) as 5 or higher on a 7-point scale (how much) using a survey developed for the project. Process Measure At the end of the project (when), 100% of students (who) will produce at minimum of 3 case studies (i.e., academic interventions with individual students from practicum site with graphed data) (what) rated by a panel of 3 faculty as showing “strong and substantial application” (how much) of evidence based practice in language and literacy.

PROJECT PERFORMANCE MEASURE EXAMPLES (1) 27 Outcome measure At the end of practicum (when), 90% of scholars (who) will have their collaboration skills (what) rated by 2 field supervisors as 6 or higher (how much) on a 7-point scale using a standard instrument developed by the project. Outcome measure By the end of Year 2 of their program (when), 100% of scholars (who) will have received ratings of 4 or higher (how much) on the masters comprehensive exams (what) based on the 5-point scale used for evaluation. Outcome measure By the end of their program (when), 100% of scholars (who) will attain at least 90% (how much) of required competencies (what) as measured by faculty evaluations of a web-based portfolio

COMMON PROBLEMS Activities are NOT performance measures If the best response is “Yes, we did that,” it is likely an activity (not a performance measure) Examples of activities: – Scholars complete required coursework – Hold an Advisory Board meeting quarterly – Establish a mentoring program – Evaluate the project Performance measures need to be measurable Examples with measurement problems: – Maintain meaningful collaborative partnerships – Provide mentors to socialize scholars to the program – Provide academic support in areas of concern to scholars 28 Taken from the Center for Evaluation and Education Policy (CEEP) presentation:

NEED ADDITIONAL INFORMATION ON WRITING PERFORMANCE MEASURES? Please contact your project officer for additional resources on developing performance measures and logic models 29

SUMMARY Projects should have a few clear objectives that explain what the project is doing to support their overall goals Each objective should have a few, specific performance measures to demonstrate how progress toward meeting the objective is being measured 30

COMPLETING ED GRANT PERFORMANCE REPORT Review the recorded webinar on Completing the ED Grant Performance Report (ED 524B) or the Annual Performance Report for Continuation Funding 31

Contact your OSEP Project Officer with any questions! 32