1. Accountability Overview Measures and Results
Annual Program Performance Report (APPR) For Teacher Preparation Programs
Accountability Overview
Measures and Results

2. Agenda
Part 1: Overview of the statue authorizing the Annual Program Performance Report (APPR) Accountability Measures.
Part 2: Summary of the Measures as defined in Rule 6A-5.066, FAC: Approval of Teacher Preparation Programs.
Part 3: Overview of the Ratings per Metric.
Part 4: Overview of the APPR 2015 Results

3. Statutes (8), (4) & (4)
Accountability Authority for the Professional Development Certification Programs, Universities and Florida Colleges, and Private Institutions

4. Statutes 1012.56(8), 1004.04(4)(a), & 1004.85(4)(b)
Criteria for continued approval for the programs must be “based upon the department’s review of performance data. The department shall review the performance data as a part of the periodic review of each school district’s professional development system required under s ”
This includes evidence of completer performance in six metrics:
Placement rate in instructional positions*
Retention rate
Student learning growth data**
Student performance by subgroup**
Teacher evaluation results
Production of completers among critical teacher shortage areas*
* Exception for Professional Development Certification Programs
** Currently not included with PDCP but will be in Spring 2018
Institutions and Universities: (4)(a), (4)(b)

5. Statutes 1012.56(8), 1004.04(4)(a), & 1004.85(4)(b)
The law authorizes State Board of Education to adopt rules for continued approval of teacher preparation programs which include:
Program review process;
Continued approval timelines; and
Performance level targets for each of the continued approval criteria (performance metrics).

6. 6A-5.066, FAC Approval of Teacher Preparation Programs
Description of the Metrics Defined in the Rule
We received stakeholder input from the following groups in developing this rule
Teacher and Leader Preparation Implementation Committee (TLPIC)
University of Florida in collaboration with all state colleges and universities across Florida
Pilot Site Visit Review Teams during
Rule Development Workshops (August 2013, June 2014 and November 2014)

7. Basic Content Areas of the Rule
Definitions
Initial approval processes
Request to Submit Form, RTS-2014
Electronic folios – described in Initial Program Approval Standards, ITP IAS-2014, EPI IAS-2014, PDCP IAS-2014
Continued approval processes
Annually submit candidate/completer data
Annual program evaluation plans (IPEP, APEP and DPEP)
Annual Program Performance Report (APPR) for each program based on the six performance metrics/targets
Performance level targets for each metric yielding score of 1 to 4
APPR summative rating score between 1.0 & 4.0

8. Placement Rate
The number of program completers reported annually by each program who were employed in a full-time or part-time instructional position in a Florida public school district in either the first or second academic year subsequent to program completion. E
Example: Completers in employed in either or
Program completers employed in a private or out-of-state P-12 school their first or second year following program completion are also included in the calculation if data are reported by the program and have been verified.
If a program provides documentation of a program completer’s death or disability, the number of program completers included in the calculation will be adjusted.

9. Retention Rate
The average number of years that program completers are employed in a full-time or part-time instructional position in a Florida public school district at any point each year in a five-year period following initial employment in either of the two subsequent academic years following program completion.
Example: completers from who were employed in either or The average number of years employed from through or through
Program completers employed in a private or out-of-state P-12 school their first or second year following program completion are also included in the calculation if data are reported by the program and have been verified.
If a program provides documentation of a program completer’s death or disability, the number of program completers included in the calculation will be adjusted.

10. Student Learning Growth
The score based on the performance of P-12 students assigned to in-field teacher preparation program completers from the previous three-year period who received a student learning growth score from the most recent academic year for which results are available.
Example: completers from , and who were employed in and taught courses related to Math (grades 3-8) and/or Reading/ELA (grades 3-10) and received a state DOE VAM score.

11. Student Performance by Subgroup
The performance of students in P-12 who are assigned to in-field teacher preparation program completers aggregated by student subgroup, as a measure of how well the teacher preparation program prepares instructional personnel to work with a diverse population of students in a variety of settings in Florida public schools.
The score is based on in-field program completers from the previous three-year period who received a student learning growth score from the most recent academic year.
Example: Completers from , and who were employed in and taught courses related to Math (grades 3-8) and/or Reading/ELA (grades 3-10) and received a state DOE VAM score. The percent that met expectation is used for this measure.
Subgroups: Free/Reduced Lunch, English Lang. Learners, Students with Disabilities, Race/Ethnicity (Asian, Black, Hispanic, Native American/ Pacific Islander, White)

12. Teacher Evaluation Results
The district’s instructional personnel evaluation rating as outlined in F.S where at least 1/3 of the evaluation must be based on the instructor’s student performance and at least 1/3 based on their instructional practice performance.
Example: Completers from , and who were employed in and received a final evaluation rating from the district for that year.
The ratings differentiate among four levels of performance as follows:
1. Highly effective.
2. Effective.
3. Needs improvement or, for instructional personnel in the first 3 years of employment who need improvement, developing.
4. Unsatisfactory.

13. Production of Graduates Among Critical Teacher Shortage Areas
A bonus score is awarded when the number of program completers in specified critical teacher shortage areas increases from the most recent year compared to the number of program completers from the previous academic year.
Example: Number of completers from compared to the number of completers in

14. Measure Ratings

15. Metric Ratings Defined in the Rule 6A-5.066
Placement Rate:
Performance Metrics
Level 4
Performance
Target (4 points)
Level 3
Target (3 points)
Level 2
Target (2 points)
Level 1
Target (1 point)
Placement Rate
(not applicable for PDCP programs per s (8), F.S.)
Placement rate is at or above the 68th percentile of all equivalent programs across the state.
Placement rate is at or above the 34th percentile and below the 68th percentile of all equivalent programs across the state.
Placement rate is at or above the 5th percentile and below the 34th percentile of all equivalent programs across the state.
Placement rate is below the 5th percentile of all equivalent programs across the state. 1 2 3 4

16. Metric Ratings Defined in the Rule 6A-5.066
Retention Rate:
Performance Metrics
Level 4
Performance
Target (4 points)
Level 3
Target (3 points)
Level 2
Target (2 points)
Level 1
Target (1 point)
Retention Rate
The average number of years employed in the 5-year period following initial placement is 4.5 years or more.
The average number of years employed in the 5-year period following initial placement is 3 years to less than 4.5 years.
The average number of years employed in the 5-year period following initial placement is 2 years to less than 3 years.
The average number of years employed in the 5-year period following initial placement is less than 2 years.

17. Metric Ratings Defined in the Rule 6A-5.066
Student Learning Growth:
Performance Metrics
Level 4
Performance
Target (4 points)
Level 3
Target (3 points)
Level 2
Target (2 points)
Level 1
Target (1 point)
Performance of Prekindergarten-12 students on statewide assessments using results of student learning growth formula per s , F.S.
The probability that the average student learning growth among students taught by program completers exceeds the expectations for those students is ≥ 95 percent.
The probability that average student learning growth among students taught by program completers exceeds the expectations for those students is < 5 percent; AND
the probability that average student learning growth among students taught by program completers falls short of the expectations for those students expectations is < 5 percent.
Not calculated.
The probability that the average student learning growth among students taught by program completers falls short of the expectations for those students is ≥ 95 percent.

18. Metric Ratings Defined in the Rule 6A-5.066
Student performance by subgroup:
Percent of the total number of subgroups, for which data were available, where the percentage of students who exceeded performance expectation within the subgroup was greater than the statewide average percent of students that exceeded performance expectation.
Performance Metrics
Level 4
Performance
Target (4 points)
Level 3
Target (3 points)
Level 2
Target (2 points)
Level 1
Target (1 point)
Student performance by subgroups data
At least 75 percent of the subgroups meet or exceed the state standard for performance.
At least 50 percent, but less than 75 percent of the subgroups meet or exceed the state standard for performance.
At least 25 percent but less than 50 percent of the subgroups meet or exceed the state standard for performance.
Fewer than 25 percent of the subgroups exceed the state standard for performance.

19. Metric Ratings Defined in the Rule 6A-5.066
Teacher evaluation results:
Performance Metrics
Level 4
Performance
Target (4 points)
Level 3
Target (3 points)
Level 2
Target (2 points)
Level 1
Target (1 point)
Results of program completers’ annual evaluations as specified in s , F.S.
At least 30 percent of the program’s completers received a highly effective rating and 90 to 100 percent of the program’s completers received either highly effective or effective ratings, and no completers were rated unsatisfactory.
Program did not meet criteria for Level 4, but at least 80 percent of the program’s completers received either highly effective or effective ratings, and no completers were rated unsatisfactory.
Program did not meet criteria for Level 3, but at least 60 percent of the program’s completers received a highly effective or effective rating and no more than 5 percent (more than one (1) for n < 20) of the program’s completers were rated unsatisfactory.
Program did not meet criteria for Level 2, 3 or 4.

20. Metric Ratings Defined in the Rule 6A-5.066
Production of critical teacher shortage areas:
Performance Metrics
Level 4
Performance
Target (4 points)
Level 3
Target (3 points)
Level 2
Target (2 points)
Level 1
Target (1 point)
Production of program completers in statewide critical teacher shortage areas, per Rule 6A , F.A.C., in accordance with Section , F.S.
BONUS ONLY, pursuant to subparagraph (3)(a)6. of this rule.
The critical teacher shortage program increased the number of program completers compared to the year before with a minimum of 2 completers in each year.

21. Metric Ratings Defined in the Rule 6A-5.066
Final Rating
Each APPR receives a summative rating score between 1.0 and 4.0 that is the average of all performance target level scores received by a program.
If the program is eligible for the bonus performance metric for the production of program completers in a statewide critical teacher shortage area, then all other performance measures are weighted as 80% and the CTS score as 20%.
The program must have three (3) or more completers in the selected cohort time period for the Placement performance metric or Retention performance metric
The program shall also have two (2) or more completers who received an annual evaluation for the Annual Evaluation performance metric.
A program that does not receive an APPR shall receive a summative rating score of 1.0 for that year.
The institution, private provider or school district has forty-five (45) business days to review the findings.

22. 2015 APPR Results

23. 2015 APPR Results

24. 2015 APPR Results

25. 2015 APPR Results

26. Thank you! APPR Accountability Questions? Email us at VAM@fldoe.org
General APPR Questions? Kimberly Pippin at