1. Ongoing Lead Evaluator
Day 1

2. Today’s Objectives
I can…go back to the basics of APPR and d to review my district’s compliance with the rules and regulations.
I can…identify areas where our district may have misconceptions or may wish to revisit decisions.

3. Warm-Up!
On a scale of 1-10, where 1 is totally out of compliance and 10 is total compliance, where would you say your district falls when it comes to following the rules and regulations of APPR?
Share with your table.

4. Corrective Action
3012-d requires an examination of APPR and score distributions. SED will have the option of imposing a Corrective Action Plan if there are significant discrepancies. Previously, Corrective Action could not impinge on anything that had been bargained. Corrective Action can now be asserted even over things that were bargained.

5. APPR Audits 51 districts in NYS 10 districts in CNY
4 districts in OCM BOCES
NYSED is informing districts this is a review process and should not be viewed as a formal audit but an opportunity to review and improve practice.

6. APPR Audits SLO’s Observation Documents TIPs/PIPs
Let’s take a look at the letter that went out to districts.

7. Corrective Action
3012-d requires an examination of APPR and score distributions. SED will have the option of imposing a Corrective Action Plan if there are significant discrepancies. Previously, Corrective Action could not impinge on anything that had been bargained. Corrective Action can now be asserted even over things that were bargained.

8. APPR Audits
A – in the column indicates there was not enough data to make it publically available.

9. APPR Audits Some Trends:
Districts that were new Focus Districts tended to be selected.
Districts that didn’t report all of their data tended to be selected.
Districts where there were big discrepancies between State Provided Growth Ratings and the reported Transition Ratings tended to be selected.
data.nysed.gov

10. APPR Reporting
A memo distributed this week indicates districts that do not submit all of the required APPR data are at risk of losing their state aid increases.
This includes original and transition ratings!
Checkpoint: Is your district submitting all of the required APPR data?

11. 10,000 Foot Overview

12. Observations, Portfolio, Survey
3012-c Points Structure
100
total points
HEDI Rating 60
Observations, Portfolio, Survey 40
Student Performance 20
State SLO/SPG 20
Local SLO

13. Teacher Observation/Principal School Visit HEDI
3012-d Points Structure
100%
Teacher Observation/Principal School Visit HEDI
100%
Overall HEDI
Scaled Scores
Observation
4 points
Student Performance
20 points
No longer do we have one 100-point configuration. We now have two separate scores that each get their own HEDI rating and get compared via the matrix (next slide).
Student Performance
HEDI

14. 3012-d Matrix for Overall Rating
Teacher Observation/Principal School Visit
Student Performance
Highly Effective
Effective
Developing
Ineffective H E D I D* \
*If a teacher is rated ineffective on the Student Performance category, and a state-designed supplemental assessment was included as an optional subcomponent of the Student Performance category, the teacher must be rated Ineffective overall.

15. 3012-d Matrix for Overall Rating
Teacher Observation/Principal School Visit
Student Performance
Highly Effective
Effective
Developing
Ineffective H E D I D*
When asked if the matrix is weighted more toward the student performance side of the system, we drew this line of symmetry to show the matrix appears to be equally weighted.
*If a teacher is rated ineffective on the Student Performance category, and a state-designed supplemental assessment was included as an optional subcomponent of the Student Performance category, the teacher must be rated Ineffective overall.

16. APPR Emergency Regulation Timeline
Which APPR Regulation is in Effect?
Emergency
Period
Transition
Period
Full
Implementation
, ,
and forward
3012-C
3012-D
3012-D
3012-D

17. APPR Emergency Regulation Timeline
What Needs to be Reported?
Emergency
Period
Transition
Period
Full
Implementation
, ,
and forward
Emergency Ratings
Original Ratings
Transition Ratings
Original Ratings
Original Ratings
During the emergency and transition periods, two ratings will need to be reported for many teachers and principals regardless of 3012-c or 3012-d status.
Transition ratings can not use any data derived from 3-8 ELA/Math Assessments or 9-12 State Provided Growth Scores

18. State Provided Growth Scores

20. Why Growth?
One way to compare student performance on tests across teachers’ classrooms is to observe student proficiency at the end of the year.
Based on this information alone, whose students had a better year?
─ Proficiency

21. Why Growth?
By adding in prior performance (blue bars), we can see how much a student grew from last year to this year. With this additional information, now whose students had a better year?
─ Proficiency
EngageNY.org

22. How to Measure Growth?
Test score scales are not comparable across grades.
Difficult to measure growth directly.
Example: Last month I had $100 in my checking account. This month I have 85 Euros. How much money have I gained or lost?
Extra credit: I have about $11 or 8€ more this month than last.
EngageNY.org

23. How to Measure Growth? Starting point matters.
Example: Her hair grew one inch last month.
So did his.
Whose hair growth seems more impressive?
Starting point matters.
Just knowing how much growth happened may not be enough. Need a system that provides fair comparisons.
EngageNY.org

24. Growth Scores Explained

25. Factors Used to Define “Similar Students” in the Growth Model
Grades 4–8 ELA/Math Teachers and their Principals
Similar Student Characteristics Used
Academic History:
Up to three years of student state exam scores, same subject
Prior-year test score, different subject
Retained in grade
Average prior achievement and range around average prior score in student’s class/course (same subject)
Student With Disability (SWD)
SWD (Yes/No)
SWD spends less than 40 percent of time in general education setting for the entire day
Percentage of SWDs in student’s class/course
English Language Learner (ELL)
ELL (Yes/No)
New York State English as a Second Language Achievement Test scores
Percentage of ELLs in student’s class/course
Economic Disadvantage (Poverty)
Poverty (Yes/No)
Percentage of students in poverty in student’s class/course
*Additional characteristics may be added in the future as available and approved by the Board of Regents.
EngageNY.org

26. Student Growth Percentiles: True or False?
A student with an SGP of 50 performed as well or better than 50 percent of similar students.
A student with an SGP of 80 must be proficient.
A student with an SGP of 20 grew less than a student with an SGP of 60.
The highest SGP that a student can receive is 99.
A student with an SGP of 75 got about 75% of test questions correct.
Extra credit: A student with an SGP of 55 in math and an SGP of 50 in ELA learned more math than ELA content this year.
True
False
EngageNY.org

27. A quiz!
Which of the following teachers gets a SPG Score?

28. A quiz! 3rd Grade Common Branch Teacher with 16 students

29. A quiz! 3rd Grade Common Branch Teacher with 16 students
3rd Grade Teachers Do Not Receive SPG Scores because there is no 2nd grade exam.

30. A quiz!
5th Grade Math Teacher with 45 students

31. A quiz! 5th Grade Math Teacher with 45 students
Yes! He/she teaches a course with a 4th-8th Grade ELA or Math Assessment and has at least 16 scores.

32. A quiz!
K-6 Building Principal

33. A quiz! K-6 Building Principal
Yes! And, if at least 30% of the students are covered by a grades 3-8 Exam, this principal would not need any SLO.

34. A quiz!
9-12 Assistant Principal

35. A quiz! 9-12 Assistant Principal
No. There are actually no reporting requirements under 3012-d for Assistant Principals.
The 9-12 Principal would receive a SPG Score.

36. A quiz!
Geometry Teacher

37. A quiz! Geometry Teacher
No. Although the results of the HS Common Core Regents are what are used for the SPG Score for the 9-12 principal, the teachers of those courses don’t receive SPG Scores themselves.

38. A quiz!
Self-Contained 12:1:1 Teacher with 4th-6th graders.

39. A quiz! Self-Contained 12:1:1 Teacher with 4th-6th graders.
Maybe. The teacher would need at least 16 scores in the NYSTP to get a SPG Score.

40. A quiz!
Could a district use a SPG Score as part of the APPR rating for everyone in the building?

41. A quiz!
Could a district use a SPG Score as part of the APPR rating for everyone in the building?
Sure. The APPR Committee could negotiate that all teachers in the building who don’t get their own SPG Score will get whatever score the principal gets.
Except that during the transition period, SPG scores may not be used in employment decisions, including TIPs and PIPs.

42. Transition Period
During the transition period, no transition scores or employment decisions, including TIPs or PIPs can be based on State Provided Growth Scores or the 3-8 ELA/Math Exams.
Checkpoint: Does your district have appropriate transition scores in place?

43. Student Learning Objectives

48. Student Learning Objectives
Considered the “Other Comparable Measures” of APPR
Used for teachers who do not have a state-provided growth score, or whose state-provided growth score does not cover at least 50% of their students.
Meant to measure teacher performance based on students demonstrating “one year’s growth in one year’s time.”

49. The 50% Rule 50% of students must be covered.
Like classes are grouped as courses.
Start with the largest course and keep adding until you get to 50%.
All students in the course must be accounted for.
If there is a state assessment at the end of the course, that must be the basis for the SLO.

50. The 50% Rule HS Math Teacher Geometry 22 Ss Math Elective A 23 Ss
Math Elective B
29 Ss
Geometry Ss
What is the total number of students?
What is 50% of that total number
Are there like courses?
How many students are in each course?
Starting with the largest courses, get to 50%
What assessment would be used?

51. The 50% Rule HS Math Teacher Math Elective A 22 Ss
Math Elective B Ss
Math Elective B Ss
Geometry Ss
What is the total number of students?
What is 50% of that total number
Are there like courses?
How many students are in each course?
Starting with the largest courses, get to 50%
What assessment would be used?
Must use both Math Elective B and then add in Math Elective A

52. The 50% Rule “Test” With your table group…
Read each of the three teacher profiles.
Identify how their Student Performance Measure will be built under 3012-d by applying the 50% rule.
Checkpoint: Are all of your teachers accurately applying the 50% Rule?

53. SLO True/False
A teacher who teaches a mix of Regents and non-Regents courses writes SLOs for Regents courses first.
All SLOs must provide some kind of baseline data, but this does not have to be a pre-test.
If a teacher goes over 50% of his students, he can pick which sections to include in his SLO and which to omit.
A teacher can adjust his/her SLO individual targets for students based on interim assessment results throughout the school year.
It’s common for teachers to have to write an SLO for more than one course/subject.
Job-alike teachers in a district must have their SLO based on the same assessment.
Common Core Regents exams have state-provided growth scores.

54. SLO True/False
A teacher who teaches a mix of Regents and non-Regents courses writes SLOs for Regents courses first.
All SLOs must provide some kind of baseline data, but this does not have to be a pre-test.
If a teacher goes over 50% of his students, he can pick which sections to include in his SLO and which to omit.
A teacher can adjust his/her SLO individual targets for students based on interim assessment results throughout the school year.
It’s common for teachers to have to write an SLO for more than one course/subject.
Job-alike teachers in a district must have their SLO based on the same assessment.
Common Core Regents exams have state-provided growth scores.

56. All SLOs Must Have Course Population Learning Content
Interval of Instructional Time
Evidence (baseline/post-assessment)
Baseline Data
Targets
HEDI Scale
Rationale
Checkpoint: Do your teachers adequately complete each of these sections of the SLO document?

57. Setting SLO Targets

69. Plan Approval Lots of Superintendent Attestations:
Back-up SLOs are in place (no need in transition period)
All targets are 1 year’s growth
All targets are reviewed and approved
Checkpoint: If your district were to be audited, would you “pass” on these 3 components?

70. Observations

71. 2 Options for Teacher Observation/Principal School Visit Subcomponent#1 #2
Teacher Observation/Principal School Visit
Teacher Observation/Principal School Visit
Independent Evaluator
Supervisor
Supervisor
Independent Evaluator
Peer Evaluator
#1 illustrates what is required.
#2 illustrates what is optional.
At Least 80%
At Least 10%
At Least 80%
At Least 10%
Not more than 10%

72. Observable Rubric Subcomponents
All observable Standards/Domains of the selected rubric must be addressed across the total number of observations.
Evaluators may select a limited number of observable rubric subcomponents for focus during a particular observation.
All non-observable Standards/Domains of the selected rubric may be observed during pre-ob. or post-ob. conferences or in natural conversations between the teacher and evaluator and incorporated into the observation score.
Points can not be given based on artifacts, unless that artifact constitutes evidence of an otherwise observable rubric subcomponent (eg: lesson plan viewed during the course of the classroom observation may constitute evidence of professional planning).
Districts may also decide how to weight the scoring of individual standards within their plans.

73. New York State Teaching Standards
Standard I
(Danielson 1)
Knowledge of Students and Student Learning
Standard II
Knowledge of Content and Instructional Planning
Standard III
(Danielson 3)
Instructional Practice
Standard IV
(Danielson 2)
Learning Environment
Standard V
(Danielson 1, 3, 4)
Assessment for Learning
Standard VI
(Danielson 4)
Professional Responsibilities and Collaboration
Standard VII
Professional Growth

74. New York State Teaching Standards
Standard I
(Danielson 1)
Knowledge of Students and Student Learning
Standard II
Knowledge of Content and Instructional Planning
Standard III
(Danielson 3)
Instructional Practice
Standard IV
(Danielson 2)
Learning Environment
Standard V
(Danielson 1, 3, 4)
Assessment for Learning
Standard VI
(Danielson 4)
Professional Responsibilities and Collaboration
Standard VII
Professional Growth
Standard III and IV represent the observable teaching standards. Standard V includes some observable areas and others that are not observable.
Not all rubrics clearly delineate what are observable items.

75. The Observation Portion
At least two observations, one of which has to be unannounced. Total amounts, frequency, direction, and communication plans are worked out locally.

76. The Observation Portion
At least one observation has to be completed by the principal or other trained administrator. At least one observation has to be completed by an impartial, independent trained evaluator. This observer cannot be assigned to the same school building as the teacher.*

77. The Observation Portion
If using peer observers:
The district chooses the peer evaluator
The peer evaluator must be trained
The peer evaluator must have been rated as H or E in the previous year

78. Teacher Observation/Principal School Visit Scoring Ranges
Minimum
Maximum
Highly Effective
3.50 to 3.75
4.0
Effective
2.50 to 2.75
3.49 to 3.74
Developing
1.50 to 1.75
2.49 to 2.74
Ineffective
1.49 to 1.74
Each observation/school visit will receive an overall 1-4 score.
Once all evaluations are complete, the different types are combined using a weighted average to produce an overall 1-4 Observation/School Visit score.
The overall Observation/School Visit score is converted to a HEDI rating based on locally determined ratings consistent with the ranges listed above.

79. Principals’ “Observation”
One observation shall be conducted by the principal's supervisor. A second observation shall be conducted by one or more impartial independent evaluators. This observer may be employed by the district but not assigned to the principal’s building (see next slide).

80. Principals’ “Observation”
A principal’s supervisor (i.e., the Superintendent) is a district employee and therefore assigned to a different BEDS code as the principal being evaluated—therefore it is possible for the same administrator to serve as both the supervisor and impartial evaluator for the purpose of school visits.
See Section H of the §3012-d APPR Guidance document for more information.

81. Observations
Checkpoint: Do you have good notes on your observations? Are you rating all of the observable standards?

82. TIP/PIP

83. Improvement Plans
The content of improvement plans will be determined by superintendent rather than through bargaining or negotiations.
Required for ineffective or developing
In place by October 1st
Include: areas in need of improvement, timeline, assessment of improvement
Should be keeping progress and conclusory notes
Checkpoint: Are all of these things happening

84. Moving Forward Checking to ensure compliance in district.
Making note of things you might wish to change in the future.
Really checking those SLOs!
Our Next Session: November 2nd (Syracuse) or November 9th (Cortland)