1. 325D Preparation of Leadership Personnel Webinar on the Annual Performance Report for Continuation Funding Presenter: Patricia Gonzalez, Ph.D. Office of Special Education Programs US Department of Education ;

2. Annual Grant Performance Report
An annual report of your performance in meeting the approved objectives of the project
Required for all active grants, including those in no cost extension (NCE)
OSEP reviews the report to determine if substantial progress has been made in order to grant continued funding or an NCE

3. Overview
Recognize strong project objectives that can be associated with high quality performance measures
Develop relevant, measurable, outcome- oriented performance measures that maximize the potential for meaningful data
Complete the ED Grant Performance Report (ED 524B)

4. Why Is This Important? High quality objectives and measures …
Make it easier for you to measure your progress
Allow you to report progress easily and quantitatively
Establish targets (both short-term/annual & long- term)
Allow OSEP staff to gather evidence of program effectiveness
Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

5. Goal – Objectives - Measures
Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

6. What are you trying to accomplish? Preferred format for objectives:
Project Objectives
What are you trying to accomplish?
Preferred format for objectives:
Begin the objective with a verb and define a desired outcome or condition
Good objectives facilitate all aspects of program implementation and evaluation

7. High Quality Project Objectives
Relevance
How relevant is the project objective to the overall goal of the program and/or the goal of your project?
Applicability
How applicable is the project objective to the specific activities that are being conducted through your particular project?
Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

8. High Quality Project Objectives
Focus
How focused is the project objective?
Measurability
Are there concepts in the project objective that lend themselves to measurement? If so, is measurement feasible?
Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

9. Project Objectives -- Examples
Recruit high quality scholars
Provide ongoing support to retain scholars in the program
Provide learning opportunities that result in scholars attaining jobs as faculty in higher education (e.g., knowledge of evidence-based practice, writing/research, teaching, and cultural competence
Develop and offer courses in assistive technology that are evidence-based

10. How are you measuring your progress in meeting your objectives?
Performance Measures
How are you measuring your progress in meeting your objectives?

11. Performance Measures
Measurable indicators used to determine how well objectives are being met.
How will progress be assessed?
How much progress will constitute success?
How will it be known if an objective or part of an objective has been achieved?
Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

12. Performance Measures
Taken from the Center for Evaluation & Educational Policy (CEEP) at Indiana University

13. Types of Performance Measures
Program
Measures established by OSEP for the Personnel Development program. These include measures established for reporting to Congress under the Government Performance and Results Act of 1993.

14. Personnel Development PROGRAM Performance Measures
#1: The percentage of Special Education Personnel Development projects that incorporate evidence-based practices into their curricula.
#2: The percentage of scholars completing Special Education Personnel Development funded programs who are knowledgeable and skilled in evidence-based practices for infants, toddlers, children, and youth with disabilities.
#3: The percentage of Special Education Personnel Development funded scholars who exit preparation programs prior to completion due to poor academic performance.
#4: The percentage of Special Education Personnel Development funded degree/certification program recipients who are working in the area(s) in which they were prepared upon program completion.
#5 (not for 325D): The percentage of Special Education Personnel Development funded degree/certification recipients who are working in the area(s) for which they were prepared upon program completion and who are fully qualified under IDEA.
#6 (not for 325D): The percentage of Special Education Personnel Development degree/certification recipients who maintain employment in the area(s) for which they were prepared for 3 or more years and who are fully qualified under IDEA.
#7: The Federal cost per fully qualified degree/certification recipient.

15. Program Performance Measures
Grantees should propose a project performance measure that addresses each program performance measure reported for the 325D program. The focus of the project measure must be the same as and clearly aligned with the program measure.
325D grantees must develop project measures that align with program measures 1, 2, 3, 4 and 7. Measures 5 and 6 do not apply to 325D. See the examples below.

16. Program Performance Measure
#1. The percentage of Special Education Personnel Development projects that incorporate evidence-based practices into their curricula. (Process Measure)
Example: Annually, one selected course syllabus will achieve a score of 90 or higher on its integration of evidence- based practice as measured by an expert panel using a rubric designed by the project. 16

17. Program Performance Measure
#2. The percentage of scholars completing Special Education Personnel Development funded programs who are knowledgeable and skilled in evidence-based practices for infants, toddlers, children, and youth with disabilities. (Outcome Measure)
Example: By year 5 of the project, 90% of scholars will demonstrate knowledge and skills in all research competencies at an “acceptable” level or higher using a rubric designed by the project 17

18. Program Performance Measure
#3. The percentage of Special Education Personnel Development funded scholars who exit preparation programs prior to completion due to poor academic performance. (Process Measure)
Example: As measured annually, no scholars (0%) will have exited the program due to poor academic or field-based performances. 18

19. Program Performance Measure
#4. The percentage of Special Education Personnel Development funded degree/certification program recipients who are working in the area(s) in which they were prepared upon program completion. (Intermediate Outcome Measure)
Example: By the end of year 5 of the project, 90% of scholars will be working in the area for which they were trained following program completion. 19

20. Program Performance Measure
#7. The Federal cost per fully qualified degree/certification recipient. (Efficiency Measure)
Example: The Federal cost is measured in Year 5 by dividing the total grant award (e.g., $250,000 x 5) by the number of scholars counted for Performance Measure #2. 20

21. Types of Performance Measures
Project
Measures that the grantee establishes to meet their project objectives
Project performance measures can address both the process of working towards an objective and the outcome related to meeting the objective
Ensure a mix of both process and outcome measures
Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

22. High Quality Performance Measures
High quality performance measures show
What will change
How much change you expect
Who will achieve the change
When the change will take place
Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

23. Project Performance Measure Examples
Process measure (e.g.) -
Project staff (who) will implement 4 (how much) targeted recruitment activities (what ) during the first year of the grant (when)

24. Project Performance Measure Examples
Outcome measure (e.g.) –
Inquiries from potential candidates (who) about the program (what) will increase 20% from previous recruitment efforts (how much) during the first year of the grant (when)

25. Project Performance Measure Examples
Outcome measure (e.g.) –
At the end of each semester (when), 100% of scholars (who) will rate their academic/psycho- social support (what) as 5 or higher on a 7- point scale (how much) using a standard instrument developed for the project

26. Project Performance Measure Examples
Outcome measure (e.g.) –
At the end of the project (when), 100% of students (who) will produce at minimum 3 scholarly products (e.g., manuscripts, presentations, preliminary exams) (what) that are rated by a panel of 3 faculty as showing “strong and substantial application” (how much) of evidence based practice in language and literacy

27. Project Performance Measure Examples
Outcome measure (e.g.) –
At the end of each course taught (when), 90% of scholars (who) will have their teaching competence of evidence based practices (what) rated by 2 faculty as 6 or higher (how much) on a 7-point scale using a standard instrument developed by the project

28. Project Performance Measure Examples
Outcome measure (e.g.) –
By the end of Year 3 of their program (when), 100% of scholars (who) will serve on at least one (how much) MA or undergraduate research committee (what)

29. Project Performance Measure Examples
Outcome measure (e.g.) –
By the end of their program (when), 100% of scholars (who) will attain at least 90% (how much) of required research competencies (what) as measured by faculty evaluations of a web-based portfolio

30. Common Problems Activities are NOT performance measures
If the best response is “Yes, we did that,” it is likely an activity (not a performance measure)
Examples of activities:
Scholars complete required coursework
Faculty meet with scholars on a monthly basis
Evaluate the project
Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

31. Common Problems Performance measures need to be measurable
Examples with measurement problems:
Maintain meaningful collaborative partnerships
Provide mentors to socialize scholars to the program
Provide academic support in areas of concern to scholars
Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

32. Need additional information on writing performance measures?
All grantees are strongly encouraged to seek training on writing performance measures.
For further information on developing performance measures and logic models, see -

33. Summary
Projects should have a few clear objectives that explain what the project is doing to support their overall goal(s)
Each objective should have a few, specific performance measures to demonstrate how progress toward meeting the objective is being measured
Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

34. Completing the ED 524B
The ED 524B is a required annual and final reporting form with specific instructions.
The form is used by all ED grants and has been approved by the Office of Management and Budget (OMB). Project Directors must follow the directions listed in the Dear Colleague letter and ED 524B Instructions provided by OSEP
Word or PDF versions of the forms are available at

35. X x

36. ANNUAL PERFORMANCE REPORTS
Reporting Period: For first year grants, the date is the beginning of the project through February 29th. For grants in years 2-5, it is the date from the end of the previous reporting period through February 29th. 02 29
20--__

37. ANNUAL PERFORMANCE REPORTS
Budget Expenditures:
Must be data or information from the business or grants office.
Report expenditures through the end of the “Reporting Period” .

38. ANNUAL PERFORMANCE REPORTS
Performance Measure Status:
This will be checked “No” for all performance reports. The APR contains data for the reporting period.
The date entered here will be the due date for your Final Performance Report, which is 90 days after the end of the grant.
Signatory must have authority to sign on behalf of the institution since the grant is from the Department to the institution and not to an individual. (This person is typically not the Project Director).

39. EXECUTIVE SUMMARY SHEET
OMB No
Exp. 02/28/2011
OMB No
Exp. 02/28/2011
EXECUTIVE SUMMARY SHEET H
*** Provide highlights of the project's activities and the extent to which the expected outcomes and performance measures were achieved during the reporting period. Do NOT include the project abstract.

40. PROJECT STATUS CHART H

41. PROJECT STATUS CHART H325 - - - - - -
Enter one of the project’s objectives; on subsequent pages, you will enter additional project objectives as submitted in your grant application.

42. PROJECT STATUS CHART
Enter the PROGRAM performance measure(s) that align with the objective immediately followed by any PROJECT measure(s) developed to address that program measure.
Enter any additional PROJECT performance measures that show you are measuring progress toward meeting the objective.

43. PROJECT STATUS CHART
Here you identify if the performance measure is a program measure, “PRGM,” or a project measure, “PROJ.”
PRGM
PROJ
Note: Program measure refers to one of OSEP’s 7 performance measures for the Professional Development Program. Project measures are unique to your grant.

44. PROJECT STATUS CHART
QUANTITATIVE DATA
Depending on your measure, enter either a raw number or a ratio and percentage. Enter the target number identified in the performance measure and then the actual data for this year. If complete data are not available for the measure, enter “999” (if no baseline) or “NA” in the “Raw Number” or “% “column (as appropriate) and provide an explanation at the bottom of the page.
Measure #1 8 6
Measure #2 10 10
100 5 10 50

45. Information to Include in the Explanation of Progress Section
Describe the data provided (e.g., what data collection methods were used, when were the data collected, how was a sample drawn, are there missing/incomplete data, what was the response rate, was a reliability measure taken). Your Project Officer should be able to understand and interpret the number in the chart from your description in this section.
What changes in the data occurred since last APR (i.e., trend)?
What activities were undertaken to achieve the targets?
If targets were not met, what are possible reasons?
How will activities that failed to meet targets be improved?

46. PROJECT STATUS CHART
QUALITATIVE DATA
If the measure requires the collection of qualitative data, then leave the “Quantitative Data” section blank.
In the Explanation of Progress section of the page, referencing the performance measure by number, report applicable qualitative data along with other information about how these data were collected, targets and activities –refer to previous slide for additional content requirements.

47. H
Provide learning opportunities that result in scholars attaining jobs as faculty in higher education
The percentage of Special Education Personnel Development projects that incorporate evidence-based practices into their curriculum.
PRGM
Annually, one selected course syllabus will achieve a score of 90 or higher on its integration of evidence-based practice as measured by an expert panel using a rubric designed by the project.
PROJ 90 90
No data are reported on program measures in the APR (change in 2011).
The project measures must align to the program measure. Both are aligned to the project objective.
Upon completion of a seminar on universal design for instruction, the mean rating across scholars on the usefulness of the seminar will be 4.5 or greater on a 5-point scale.
PROJ
4.5
4.0 47

48. H
Provide learning opportunities that allow scholars to succeed as faculty in higher education.
Explanation of Progress (Include Qualitative Data and Data Collection Information)
(1b) We have attached the instrument developed for this purpose to Section C. There are 10 items in the rubric; each can achieve a score of 0-10, so a score of 90 reflects a reduction of 10 points across the items. Using this tool, the number in the chart is the mean rating by 2 faculty members at another campus of our university. The course selected for review was a course on Issues in Special Education. Based on their feedback, we updated references on progress monitoring.
(1c) Since this grant is designed with a focus on universal designs for instruction, a required seminar on this topic was identified for more in-depth evaluation. The seminar was offered during the Fall 2010 semester. All students in the course completed the rating. Given that the target was not met, students will be interviewed to see how the seminar might be improved for future scholars, with the goal of meeting the “usefulness” score target of 4.5 the next time the class is offered and evaluated. 48

49. Final Page of the Report
Section B: Refer to the instructions for Section B with the ED 524B
Section C: Refer to the instructions for Section C with the ED 524B—put attachments here (e.g., recruitment material, evaluation instruments, journal articles)

50. Section B – Budget Information
This section is never blank!
Actual expenditures for reporting period
Provide an explanation if you did not expend funds at the expected rate.
Describe any significant changes to your budget resulting from modifications to project activities.
Describe any changes to your budget that affected your ability to achieve your approved project activities and/or project objectives.

51. Section B – Budget Information
Do you expect to have any unexpended funds at the end of the current budget period? (Explain why, provide an estimate, and indicate how you plan to use the unexpended funds [carryover] in the next budget period.)
Describe any anticipated changes in your budget for the next budget period that require prior approval from the Department.

52. Section C – Additional Information
If applicable, please provide a list of current partners on your grant and indicate if any partners changed during the reporting period. Please indicate if you anticipate any change in partners during the next budget period. If any of your partners changed during the reporting period, please describe whether this influenced your ability to achieve your approved project objectives and/or project activities.
Describe any changes that you wish to make in the grant’s activities for the next budget period that are consistent with the scope and objectives of your approved application.

53. Section C – Additional Information
If you are requesting changes to the approved Project Director and/or to other approved key personnel with a proposed effective date during the remainder of the current or the next budget period, please indicate the name, title and percentage of time of the requested key personnel. Attach a resume or curriculum vitae for the proposed key personnel when you submit your performance report.
Note: Do not report on any key personnel changes that were already made during the current or previous budget period(s). Departmental approval must be requested and received prior to making key personnel changes.
Provide any other information about your project including unanticipated outcomes or benefits

54. Submitting the ED 524B
Submit the ED 524B at (there are instructions for using G5 in the continuation packet)
A signed ED 524B Cover Sheet must be scanned and ed in PDF format to your Project Officer.
Special cases require regular submission of the 524B and the signed cover sheet in PDF format to your Project Officer rather than submission through G5 –
Final reports and APRs for grants in their last performance period or in no-cost extension
Grants that were front-loaded (forward-funded) last year –ALL D GRANTS WERE FRONT LOADED LAST YEAR

55. Contact your OSEP Project Officer with any questions!