1. Annual Professional Performance Review (APPR) Updates for Middle Level Liaisons
Alexander Trikalinos, Office of Educator Quality and Professional Development
November 30, 2018

2. APPR Timeline

3. Timeline New York State’s Evaluation System
2010:
Governor signs Chapter 103 of the Laws of 2010; adding §3012-c, which establishes a comprehensive evaluation system for teachers and principals, effective July 1, 2010.
USDE announces that New York is selected for a RTTT award of approximately $700M. :
First year of State-provided growth score results for all 4-8 ELA and math teachers and their building principals.
Evaluations for teachers and principals are conducted in some NYS districts (e.g., School Improvement Grant and Teacher Incentive Fund recipients).
Evaluation Law is revised. Governor signs the bill into law on March 27, 2012 (Chapter 21 of the Laws of 2012). Board of Regents adopts emergency regulations to conform to the major 2012 legislative changes. :
All NYS districts must have an approved APPR plan by January 17, 2013 or risk state aid increases.
Evaluations for teachers and principals are done in all districts except for NYC. NYC is required by law to have a State-imposed evaluation plan.
Legislature further amends the Evaluation Law (Part A of Chapter 57 of the Laws of 2013). :
Second year of evaluations for all districts in NYS, except NYC. First year for NYC.
Legislature further amends the Evaluation Law (Chapter 56 of the Laws of 2014)

4. Timeline New York State’s Evaluation System (cont.):
Governor signs Chapter 56 of the Laws of 2015, establishing a revised evaluation system for teachers and principals (Education Law §3012-d).
All districts are required to have an approved APPR plan under the new statute by November 15, 2015 or to have an approved Hardship Waiver. :
18% (n=122) of districts have approved plans under Education Law §3012-d; 82% (n=567) remain under Education Law §3012-c with an approved Hardship Waiver.
All districts must have an approved APPR plan under Education Law §3012-d by December 31, 2016.
At its December meeting, the Board of Regents adopts a transition period during which time the results of the grades 3-8 ELA/math State assessments and any State-provided growth scores are to be used for advisory purposes only. Separate transition evaluations that exclude these measures will be provided to affected educators. :
First full year of implementation of Education Law §3012-d. :
Second full year of implementation of Education Law §3012-d.
Commissioner’s APPR Survey released to field.
On November 5, 2018, Board of Regents announces additional one-year extension of APPR Transition Period (through June 2020).
Regulations will be brought to Board of Regents at its December meeting.

5. Education Law §3012-d APPR Components

6. Education Law §3012-d Components of the APPR Evaluation System
Evaluations include educator practice and student learning measures
Measures result in a single overall educator effectiveness rating
Required Student Performance Measures
State-provided growth on State assessments or Student Learning Objectives
Student Learning
Required
Principal/ Administrator Observation
Supervisor/ Administrator School Visits
Educator Practice
Overall APPR Rating
Overall annual evaluation HEDI rating based on both category ratings, as applied to the evaluation matrix
Optional Student Performance Measures
Student growth – rigorous and comparable across classrooms/grade configurations and/or programs
&
Student Performance Category Rating
Combined required and optional subcomponents, per weighting indicated in approved APPR plan.
Teacher Observation/Principal School Visit Category Rating
Evidence-based observations/school visits.
Independent Evaluator Observation/School Visits
Optional
Peer Observation/School Visits

7. Education Law §3012-d Components of the APPR Evaluation System
The overall APPR rating is determined by the statutory matrix:
Observation/School Visit
Highly Effective (H)
Effective
(E)
Developing
(D)
Ineffective
(I)
Student Performance H E D
Effective (E)
Developing (D) I
Ineffective (I) D*

8. Overview of APPR Transition Period Regulations (2015-16 through 2018-19 school years)

9. APPR Transition Period Overview
At its December 2015 meeting, the Board of Regents adopted regulations creating the APPR Transition Period.
Provide for a four year transition period for annual professional performance reviews (APPRs) while the State completes the transition to higher learning standards.
During the transition period, transition scores and HEDI ratings will replace the scores and HEDI ratings for teachers and principals whose HEDI scores are based, in whole or in part, on State assessments in grades 3-8 ELA or mathematics  (including where State-provided growth scores are used) or on State-provided growth scores on Regents examinations.
For purposes of public reporting of aggregate data and disclosure to parents on request the original composite score and rating and the transition composite score and rating must be reported with an explanation of such transition composite score and rating.

10. APPR Transition Period Overview
For teachers and principals, the regulations:
Ensure that there will be no consequences for teachers and principals related to 3-8 ELA and mathematics state assessments and no growth score on Regents exams until the start of the school year.
Prohibit the use of results from the 3-8 state assessments for use in evaluating the performance of individual teachers, principals or students.

11. APPR Transition Period Overview
During the APPR Transition Period, the Department has:
Approved APPR plans for all school districts and BOCES in New York State, including the rubrics and assessments used as part of those plans.
Adopted Next Generation Learning Standards.
Changed the State assessment program in grades 3-8 ELA and math.
Developed and received approval from the United States Department of Education of its federal ESSA plan.
Worked with stakeholders and the field to collect feedback on current APPR requirements, including through the Commissioner’s APPR Survey.
Launched two APPR Workgroups – the Assessment and Evaluation Workgroups to create recommendations for reforming the evaluation system.

13. Background
This survey was developed to begin a collaborative conversation with teachers, school leaders, and district administrators on revising New York’s current APPR system.
On February 6, 2018, all public school teachers, as well as school and district leaders, received an invitation to complete a survey about current teacher and principal evaluation requirements and what an ideal system could look like.
The survey was designed to collect feedback from educators in four areas:
The overall purpose and appropriate use of evaluation to support continuous improvement;
School-based factors educators believe they have an influence on;
The inclusion and relative weight of different types of evaluation measures; and
The frequency and number of instances of each measure.
(Read slide)

14. Survey Responses Overview
The survey closed on March 30, 2018.
21,669 stakeholders completed the survey.
Survey takers included:
17,935 teachers
K-5: 44%
6-8: 32%
9-12: 38%
1,200 principals
2,534 others
Includes assistant principals, superintendents, assistant superintendents, etc.
This survey was intended to gather feedback from the field and is not an evaluative instrument. Therefore, we did not conduct a formal field test and do not have validity or reliability statistics to report. However, the survey was designed around accepted principles of high quality survey development, including respondent effort and cognitive load, question wording, order, and format, and survey structure.  
Staff in the program office were trained through USED’s Equitable Access to Support Network on the best practices in the development of feedback surveys. No action will be taken to revise the evaluation system based on the results of the survey. This is just a baseline data collection for us to use with stakeholders moving forward.
** Teacher grades: teachers were able to select all applicable grades, which is why it is greater than 100%

15. Survey Response Demographics
As you can see, educators who completed this survey had many overall years of experience as an educator. **Note that with the grades taught/overseen, survey responders could select all that were applicable (so does not add up to 100%)
Years of Overall Experience as an Educator
Grades Taught/Overseen

16. Survey Response Regional Demographics*
All educators
Actual numbers:
Buffalo: 3167
Capital: 2078
Long Island: 3092
NYC: 4142
Rochester/Southern Tier: 3197
Syracuse/North Country: 2879
Yonkers: 3091
*Each area listed above includes respondents from all school districts within that geographical region.

17. Survey Results on Teacher Evaluation

18. Key Takeaways Teacher Evaluation Survey
Most teacher respondents believe they have a moderate or high impact on all of the factors we identified:
Student Achievement
Student Growth
Social/Emotional Well-Being of Students
Professional/Soft Skill Development
Parent/Community Engagement
School Culture/Climate
However, this did not always mean they believe measures related to those factors should play a significant role in their evaluation.
Although teachers believe they have a significant impact on student achievement/growth, they did does not necessarily want those measures to have evaluative significance
A majority of teacher respondents indicated that they feel as though they have a high or moderate impact on student achievement (86%) and student growth (88%)
However, 49% of teachers believe student achievement should have a high or moderate significance in their evaluation and 69% of teachers believe student growth should have a high or moderate significance in their evaluation.

19. Key Takeaways Teacher Evaluation Survey
Measures of Student Learning
Most teachers believe they have a moderate or high impact on student achievement and growth.
Most teachers also believe measures of student growth could have a moderate to high significance on their annual evaluation.
While teachers generally agreed that student growth could be a factor in their evaluation, they view a statistical growth model much less favorably.
Teachers of courses/subjects where a school/district-wide student performance measure is currently used for evaluation often commented that they do not want to be evaluated based on students who they do not teach.
- Interesting to note that principals were more in favor of a statistical growth model (51% high/moderate significance for teacher evaluation as opposed to 30% for teachers)

20. Key Takeaways Teacher Evaluation Survey
Observations
Teachers overwhelmingly favor observations by principals or other trained administrators.
Slightly more than half of teachers also believe that observations by trained peer observers could have a moderate or high significance in their evaluations.
Observations by impartial, independent evaluators were the least popular option.
Many teachers commented that they would like to receive more frequent feedback on their classroom observations.
Teachers also commented on the importance of having evaluators with instructional expertise.

21. Key Takeaways Teacher Evaluation Survey
Other Evaluation Measures
The ‘least popular’ evaluation measures for teacher evaluation included:
Student surveys
Parent surveys
Observations by impartial, independent evaluators
Many teachers believe that a structured review of artifacts and/or goal setting aligned to Teaching Standards should be an allowable measure for their evaluation.
Teachers also expressed a desire for more focus on Standards 6 and 7 of the NYS Teaching Standards (“Professional Responsibilities and Collaboration” and “Professional Growth”) and for self-assessment to be an allowable option.

22. Key Takeaways Teacher Evaluation Survey
Professional Learning
A review of the survey results and related comments reveals that additional attention should be paid to providing professional learning opportunities related to the evaluation system for stakeholders at all levels:
Only 29% of teachers report being very familiar with APPR requirements, and 55% of teachers report being moderately familiar.
Comments reveal a concern on the part of some teachers about whether evaluators possess the necessary familiarity with their pedagogy and/or content to provide high quality, relevant feedback aligned to the Teaching Standards.
Comments reveal that teachers sometimes perceive things as required or impermissible “under the law” when those things are actually the result of local decision-making (e.g., being evaluated based on students who they don’t teach; observations being conducted infrequently, etc.)
- Interesting to note that principals were more in favor of a statistical growth model (51% high/moderate significance for teacher evaluation as opposed to 30% for teachers)

23. Further Inquiry Teacher Evaluation Survey
When asked about other measures they might want to be part of their evaluations, teachers indicated that an ideal system might include:
Student attendance and parental involvement as a mitigating factor in computing student performance metrics
Inclusion of a self-assessment component
Extra-curricular/school community involvement
Professional development participation
Evaluations and/or components of the evaluation should not necessarily be annual for seasoned/demonstrably effective teachers
Principals also indicated that they would like to see the same factors, with the addition of:
Attendance of the teacher
Commitment to the profession
Relationships with students, parents, and the school community
It is important to note that some of these “additional” measures (e.g. self-assessment, community involvement, PD, attendance, commitment, relationships) could be addressed through the existing observation process if evaluators are trained on how to collect evidence related to them.

24. Survey Results on Principal Evaluation

25. Key Takeaways Principal Survey
Similar to teachers, most principals believe they have a moderate or high impact on all of the factors we identified:
Student Achievement
Student Growth
Social/Emotional Well-Being of Students
Professional/Soft Skill Development
Parent/Community Engagement
School Culture/Climate
Also similar to teachers, this did not always mean they believe measures related to those factors should play a significant role in their evaluation.
A majority of principal respondents indicated that they feel as though they have a high or moderate impact on student achievement (82%) and student growth (82%)
However, 64% of principals believe student achievement should have a high or moderate significance in their evaluation and 75% of principals believe student growth should have a high or moderate significance in their evaluation. Should be noted that principals, on average, believe that student growth metrics should have a greater significance in their evaluation than teachers.

26. Key Takeaways Principal Evaluation Survey
Measures of Student Learning
Similar to teachers, most principals believe they have a moderate or high impact on student achievement and growth.
Also similar to teachers, most principals believe measures of student growth could have a moderate to high significance on their annual evaluation.
Different than teachers, most principals believe that measures of student achievement could also have a moderate to high significance on their annual evaluation.
Observations: supervisor/trained admin: 82% high/moderate significance; independent evaluators: 42% high/moderate significance; peer observers: 51% high/moderate significance -

27. Key Takeaways Principal Evaluation Survey
Other Measures of Effectiveness
The ‘least popular’ evaluation measures for principal evaluation included:
Student surveys
Parent surveys
Observations by impartial, independent evaluators
Some principals indicated that a survey of their staff with questions about the instructional support and quality of the feedback that they receive should be an allowable measure for their evaluation.
Principals also expressed a desire for more focus on Standards 2, 3, and 4 of the 2008 ISLLC Standards (school culture, school safety, and collaboration respectively) and for self-assessment to be an allowable option.
Least popular based on responses for “no significance”: Student Surveys: 16.51%; Parent Surveys: 15.03%; Observations by impartial, independent evaluators: 14.46% -

28. Further Inquiry Principal Evaluation Survey
When asked about other measures they might want to be part of their evaluations, principals indicated that an ideal system might include:
School climate measures
Inclusion of a self-assessment/reflection component with personal goal-setting
Long-range planning/school goal-setting
School safety
State report card data (graduation rates, proficiency/passing rates on State/Regents exams, etc.)
Additionally, the inclusion of Assistant Principals in the evaluation process was also mentioned for consideration.

29. APPR Workgroups

30. APPR Workgroups Overview
Two APPR Workgroups have been convened – the Assessment and Evaluation Workgroups.
The Evaluation Workgroup will focus on the framework for evaluation, and the discussion will include, but is not limited to: observation, student performance, and other measures of educator effectiveness.
The Assessment Workgroup will explore the use of innovative and authentic assessments (e.g., performance-based assessment, portfolio assessments, etc.) with relevant national experts to determine best practices and requirements for the use of assessment as part of teacher and principal evaluation, including how these assessments can pinpoint student understanding in meaningful ways and provide educators with valuable information to inform their instruction.

31. APPR Workgroups Overview
Each Workgroup includes approximately 60 members.
Workgroup members are primarily teachers and principals with the remainder being administrators and representatives of statewide stakeholder organizations.
Workgroup members come from every region of the State and represent educators from all content areas and grade levels.
The goal of each Workgroup is to create recommendations for the Department and Board of Regents on what an ideal evaluation system could look like for New York educators.

32. APPR Workgroups Overview
Each Workgroup will meet six times between November and March.
The final recommendations from each Workgroup will be presented to the Board of Regents in the early spring of 2019.

34. Questions. Contact the office: EducatorEval@nysed
Questions? Contact the office: Contact me: