Teacher Observations: An Integrated Approach Module 6 Dr. Regina Cohn Dr. Robert Greenberg

Instructional Planning-Delivery-Reflection move the cogs in a 100% system 2 Teacher Evaluation 60% State Test of Student Achievement 20% Locally Developed Measures of Student Achievement/SLOs 20%

Module 6 Training Objectives Understand the connections between teacher evaluation and student learning outcomes Advance our common language around teacher evaluation and student learning outcomes Understand expectations for evidence collection and assessment of skills for teacher evaluation Understand how to interpret evidence against criteria to assess teacher practice Collaborate with colleagues 3

Working Agreement Be present: minds and hands on all day Respect time boundaries Recognize the need for quiet while working Use electronics respectfully and appropriately when prompted Return to large group attention when signaled 4

TLS Evidence Rubric 5

OAR(S) Objective Aligned Representative Sufficient

Evidence is a Factual Reporting of Events It includes teacher and student actions and/or behaviors Teacher presented the content from the front of room. It includes statements made and questions posed by the teacher and the students “Bring your white boards, markers and erasers to the carpet and sit on your square.” It includes artifacts prepared by the teacher, students, or others. Task cards, journals, lesson plans, etc. It includes quantifiable information about time, student participation, resource use, etc. 9:14 – 9:29- Warm-up. 8 of 22 Ss finished at 9:20, sat still until 9:29 It includes an observed aspect of the environment Desks were arranged in groups of four 7

Trends in January Evidence Samples A preponderance of the submitted evidence samples contained Summary statements “Provided relevant examples and scaffolding to discuss how we "measure your foot" or "blood pressure" driving home the point that we need "different tools for different problems" In this way the Clinometers and meter stick were introduced before directions were given for outside task.” Conclusions/judgments “Students were engaged during small group activities.” A lack of student evidence  T: Who has a prediction?  T: What is your method?  T: Why did you use tangent? 8

Trends in January Evidence Samples Vague information “Teacher circulates” Misalignment to the criteria SCLVM—Summaries, Conclusions/Judgments, Lack of Evidence, Vagueness, Misalignment 9

Evidence Exemplars From the Field Review the Evidence Exemplars Use OAR(S) to examine and review the exemplar. Discuss with your table mates. 10

Evidence Analysis With an elbow partner examine the evidence collection activity sheet Identify the reason why the evidence is problematic E.g., “teacher circulates” is vague because it does not provide a context. Is the teacher monitoring student understanding or student behavior? Does the teacher talk with students? If so, what is the conversation? How can an evaluator determine the level of effectiveness of the teacher’s walking among the students? Be prepared to share your work with the larger group Use SCLVM as a guide 11

Expectations for Demonstrating IRR before Turn-Key Training “Other” expectation: IRR Achieving inter-rater reliability is a process… GOAL: to demonstrate acceptable IRR after calibration March/April: First official assessment of the observation process, receive feedback on three areas of evidence and trends in accuracy April/May: Demonstrate proficiency in the 3 evidence areas [objectivity, alignment, representativeness], receive feedback on evidence and trends in accuracy May/June: Demonstrate proficiency in the 3 evidence areas and in at least 2 of the 3 accuracy measures 12

Training Platform For training purposes we will be using a platform developed by True North Logic Please log on to the following site: 13

Log-In 14 Enter your user name Enter your password Click to Log-In

Welcome to the Home Page 15 Select “Observations:”

Starting the Observation 16 Select the teacher Select the rubric

Observation Practice: Pre-Observation Elements 1, 2, 4, 6, 9, IRR Individually Review the lesson plan and pre-conference notes for Ms. Rios Align the evidence to the appropriate elements/indicators [smallest grain size] Check your work with a partner who is using the same rubric Did you align your evidence to the same element/indicator? Discuss your rationale for your alignment Be prepared to share some examples of your aligned evidence collection 17

Observation Practice: Classroom Observation Individually Watch the video Collect evidence for the observable elements/indicators of your rubric Align your evidence to the appropriate elements/indicators Check your work with a partner who is using the same rubric Did you capture similar evidence? Did you align your evidence to the same elements/indicators? Discuss your rationale for your alignment. Be prepared to share some examples of your aligned evidence collection 18

Video Bright/Rios 19

Observation Practice: Post Observation Elements 1, 2, 4, 6, 9, IRR Individually Review the teacher’s reflection and post-observation notes Align the evidence to the appropriate elements/indicators Check your work with a partner who is using the same rubric Did you align the evidence to the same elements/indicators? Discuss your rationale for your alignment Be prepared to share some examples of your aligned evidence collection 20

Observation Practice: Scoring Individually Score the teacher’s practice at the element/indicator level for NYSUT Standards III and IV ASCD Standards 2 and 3 Be sure your judgments are based on your aligned evidence Reference the language of the rubric to determine the level of performance 21

SLO Information 100-Point Evaluation System Learning Content Evidence Student Population, Baseline, and Target(s) HEDI Interval and Rationale Participant SLO Sharing NY SLO Development Guide Why SLOs? Assignment, Survey, and Questions 22

Multiple Hats for the Day Throughout the day, we are engaging in the work at multiple levels: Learner: What is the content and rationale for an SLO? Teacher: How does this work support and impact teachers? District: What decisions need to be made and by whom? Trainer: How will I convey the importance and content of this work to districts, schools, and other stakeholder groups? 23

EBOP, etc. 20 Growth 20 Local 100 Student Learning Objectives (focus of this training) 100-Point Evaluation System for Teachers

1. Assess and identify priorities and academic needs. 2. Identify who will have State-provided growth measures and who must have SLOs as “comparable growth measures.” 3. Determine District rules for how specific SLOs will get set. 4. Establish expectations for scoring SLOs and for determining teacher ratings for the growth component. 5. Determine District-wide processes for setting, reviewing, and assessing SLOs in schools. 5 District Decisions by 5/30/2012 3/01 4/16 5/30 From NY DOE engageny.org Introduction to SLO Deck

Discussing District Decisions Decision 1: Assess and identify district priorities and academic needs. Decision 5: Determine district-wide processes for setting, reviewing, and assessing SLOs in schools. Recognize how district priorities and academic needs can inform learning content selections. Determine at what level your district(s) should consider resting the decision for learning content selections (district, school, teacher, etc.). 26

100-Point Evaluation System: State 20% Three types of teachers: 27

Discussing District Decisions Decision 2: Identify who will have State- provided growth measures and who must have SLOs as “comparable growth measures.” Using the three types of teachers, discuss who will have SLOs as “comparable growth measures.” Refer to the SLO materials provided on EngageNY.org as needed. Policy/State questions: 28

NYSED SLO Framework A student learning objective is an academic goal for a teacher’s students that is set at the start of a course. It represents the most important learning for the year (or, semester, where applicable). It must be specific and measurable, based on available prior student learning data, and aligned to Common Core, State, or national standards, as well as any other school and district priorities. Teachers’ scores are based upon the degree to which their goals were attained. Source: Page 4 of Guidance on the New York State District-Wide Growth Goal Setting Process: Student Learning Objectives 29

NYSED SLO Framework All SLOs MUST include the following eight basic components: Student PopulationWhich students are being addressed? Learning Content What is being taught? CCSS/National/State standards? Will this goal apply to all standards applicable to a course or just to specific priority standards? Interval of Instructional Time What is the instructional period covered (if not a year, rationale for semester/quarter/etc)? Evidence What assessment(s) or student work product(s) will be used to measure this goal? Baseline What is the starting level of learning for students covered by this SLO? Target(s) What is the expected outcome (target) by the end of the instructional period? HEDI Criteria How will evaluators determine what range of student performance “meets” the goal (effective) versus “well-below” (ineffective), “below” (developing), and “well-above” (highly effective)? RationaleWhy choose this learning content, evidence and target? 30

Student Population and Learning Content Team Activity: ( As a team, discuss and select: Student Population – These are the students included in the SLO. Learning Content – a particular course name the course’s source of standards (Common Core, etc.) the exact items (indicators, standards, etc.) to be taught, learned, and assessed. 31

Learning Content This is the content to be taught in the SLO. What is being taught? CCSS/National/State standards? Will this goal apply to all standards applicable to a course or just to specific priority standards? Task: Identify the course name and source of standards (Common Core, national, state, local) associated with this SLO, and specify the exact standards, performance indicators, etc., that will be taught, learned, and assessed. 32

Learning Content – Training SLO Training SLO Context Course: 7 th Grade Visual Arts Standards Source: New York State Learning Standards Standard 1 – Creating, Performing, and Participating in the Arts Standard 3 – Responding to and Analyzing Works of Art. Standard 4 – Understanding the Cultural Dimensions and Contributions of the Arts Performance Indicators: 1c, 1d, 3a, 3c, 4c Common Core Writing Standards for Literacy in …Technical Subjects: WHST.6-8.4, WHST

Learning Content – Training SLO (1c) [Students] use the elements and principles of art to communicate specific meanings to others in their art work. (1d) During the creative process [students ] reflect on the effectiveness of selected mediums or techniques to convey intended meanings. (3a) [Students] discuss and write their analyses and interpretations of their own works of art and the art of others, using appropriate critical language. (3c) [Students] compare the ways ideas and concepts are communicated through visual art with the various ways that those ideas and concepts are manifested in other art forms. 34

Learning Content – Training SLO (4c) [Students] create art works that reflect a particular historical period of a culture. (WHST.6-8.4) [Students] produce clear and coherent writing in which the development, organization, and style are appropriate to task, purpose, and audience. (WHST.6-8.8) [Students] gather relevant information from multiple print and digital sources, using search terms effectively; assess the credibility and accuracy of each source; and quote or paraphrase the data and conclusions of others while avoiding plagiarism and following a standard format for citation. 35

Interval of Instructional Time This is the timeframe within which the learning content will be taught. (This is generally one academic year unless the course is set as a semester, quarter, etc.) Interval of Instructional Time Task: Specify when the teaching for this learning content will begin and end. (Rationale is required if less than the typical year-long interval is set.) 36

37 Student Population (The who- All students) Learning Content (The what) Interval of Instructional Time (timeframe for learning content to be taught) Evidence (proof of what will be used) Baseline (How students perform prior to target)

38 Target(s): (The growth goal) HEDI (Criteria and description) Highly effective (18-20 points)* Effective (9-17 points)* Developing (3-8 points)* Ineffective (0-2 points)* Rationale (Why did I make these selections?)

Discussing District Decisions Decision 3: Determine district rules for how specific SLOs will get set. Review the NYSED Assessment Decision Chart to assist in planning. Decide on assessments to be used with SLOs. 39

Evidence These are the assessments used for determining students’ levels of learning. Task: As a team discuss what pre- and summative assessments best measure the selected learning content for the course and standards/performance indicators you selected. (If none is available for your learning content, describe the characteristics of a high-quality and valid assessment for the identified learning content, how it would be developed, by whom, when, etc.?) (Districts and BOCES must verify comparability and rigor for any assessments they develop.) 40

Evidence – Training SLO (Summative Rubric) 41 4 points3 points2 points 1 point (0 points for no attempt) Creative Task (1c) Use of elements (row score x 3) 3 or more elements used that strongly reflect stated viewpoint 2 elements used that strongly reflect stated viewpoint 1 element was used that strongly reflects stated viewpoint Attempt(s) made, but no elements were used that strongly reflect stated viewpoint Creative Task (1c) Use of principles (row score x 3) 3 or more principles used that strongly reflect stated viewpoint 2 principles used that strongly reflect stated viewpoint 1 principle used that strongly reflects stated viewpoint Attempt(s) made, but no principles were used that strongly reflect stated viewpoint Task 2 (WHST.6-8.8) Gather relevant source information (row score x 1) Student does both of the following for three sources:  describes convincingly why sources are credible and accurate  cites sources clearly following appropriate conventions Student does both of the following for three sources:  describes convincingly why sources are credible and accurate  cites sources generally following appropriate conventions Student does both of the following for three sources:  describes generally why sources are credible and accurate  cites sources generally and/or clearly following appropriate conventions Attempt(s) made, but one or neither of the following takes place:  describes generally why sources are credible and accurate  cites sources generally and/or clearly following appropriate conventions Note on Rubric: Each student’s work will be scored by two other district middle school visual arts teachers using the following rubric, with both teachers’ scores being averaged to yield the final student scores.

Evidence – Training SLO (Summative Rubric) 42 4 points3 points2 points 1 point (0 points for no attempt) Task 2 (WHST.6-8.8) Use relevant source information (row score x 1) Data and conclusions for three sources are given where both of the following take place:  data and conclusions are precise  sources strongly support the viewpoint Data and conclusions for three sources are given where both of the following take place:  data and conclusions are precise  sources generally support the viewpoint Data and conclusions for three sources are given where both of the following take place:  data and conclusions are general  sources generally and/or strongly support the viewpoint Attempt(s) made, but one or neither of the following for three sources takes place:  data and conclusions are general  sources generally and/or strongly support the viewpoint Task 3 (1d) How well does your medium/ technique convey your viewpoint? (row score x 2) Student does all of the following:  cites specific reasons for selecting medium/technique  connects at least one aspect of the medium as supporting the viewpoint,  acknowledges at least one limitation of the medium/technique Student does two of the following:  cites specific reasons for selecting medium/technique  connects at least one aspect of the medium as supporting the viewpoint,  acknowledges at least one limitation of the medium/technique Student does one of the following:  cites specific reasons for selecting medium/technique  connects at least one aspect of the medium as supporting the viewpoint,  acknowledges at least one limitation of the medium/technique Attempt(s) made, but did none of the following:  cites specific reasons for selecting medium/technique  connects at least one aspect of the medium as supporting the viewpoint,  acknowledges at least one limitation of the medium/technique

Evidence – Training SLO (Summative Rubric) 43 4 points3 points2 points 1 point (0 points for no attempt) Task 4 (3c) Communicati ng in other art forms (row points x 2) Student compares at least one unique strength and corresponding impact of each of the three other art forms Student compares at least one unique strength and corresponding impact of each of two other art forms Student compares at least one unique strength and corresponding impact of one other art form Attempt(s) made, but student does not compare at least one unique strength and corresponding impact of one other art form Task 5 (3a, WHST.6-8.4) Written analysis and interpretation of own work (row points x 3) Student does both of the following:  identifies how three elements and/or principles contribute toward (or away from) viewpoint, citing relevant evidence from work  discusses two interpretations of viewpoint citing relevant evidence from work Student does both of the following:  identifies how two elements and/or principles contribute toward (or away from) viewpoint, citing relevant evidence from work  discusses one or two interpretations of viewpoint citing relevant evidence from work Student does both of the following:  identifies how one element and/or principle contribute toward (or away from) viewpoint, citing relevant evidence from work  discusses one or two interpretations of viewpoint citing relevant evidence from work Attempt(s) made, but does only one or neither of the following:  identifies how one element and/or principle contribute toward (or away from) viewpoint, citing relevant evidence from work  discusses one or two interpretations of viewpoint citing relevant evidence from work

Evidence – Training SLO (Summative Rubric) 44 4 points3 points2 points 1 point (0 points for no attempt) Task 6 (WHST.6-8.8) Gather and use relevant source information (row score x 1) Student does all of the following for the critique:  selects a published critique  paraphrases critique’s data and conclusions precisely  states own conclusion and reason clearly and with relevant evidence  cites source clearly following appropriate conventions Student does all of the following for the critique:  selects a published critique  paraphrases critique’s data and conclusions generally or clearly  states own conclusion and reason clearly and with relative evidence  cites source generally or clearly following appropriate conventions Student does all of the following for the critique:  selects a published critique  paraphrases critique’s data and conclusions generally or clearly  states own conclusion and reason clearly (no relative evidence)  cites source generally or clearly following appropriate conventions Attempt(s) made, but one or neither of the following takes place:  selects a published critique  paraphrases critique’s data and conclusions generally or clearly  states own conclusion and reason clearly (no relative evidence)  cites source generally or clearly following appropriate conventions Task 7 (3a, WHST.6-8.4) Comparison of own work to published artist’s work (row score x 2) Student does both of the following:  states clearly how multiple different approaches are taken  states clearly how multiple similar approaches are taken Student does both of the following:  states clearly how multiple different approaches are taken  states clearly how one or more similar approaches are taken Student does both of the following:  states clearly how one different approaches are taken  states clearly how one or more similar approaches are taken Attempt(s) made, but does only one or neither of the following:  states clearly how one different approaches are taken  states clearly how one or more similar approaches are taken

Discussing District Decisions Decision 4: Establish expectations for scoring SLOs and for determining teacher ratings for the growth component. Decision 5: Determine district-wide processes for setting, reviewing, and assessing SLOs in schools. Discuss district implications for establishing baselines and setting targets for the student population. (Considerations include common target expectations, where decisions for these selections will rest, etc.) 45

Baseline and Target(s) 46 Baseline This is the level of students’ knowledge and skill in the targeted learning content at the beginning of the interval of instructional time. Target(s) This is the level of knowledge and skill that students are expected to achieve at the end point of the interval of instructional time.

Baseline and Target(s) 47 Team Activity: Baseline: Describe how students performed on the identified pre-assessment(s) for the learning content. (Actual baseline scores for each student are required.) Target(s): Define numerical growth goals for student performance on identified summative assessment(s) which measure student knowledge and skill in the learning content. (Actual final scores for each student are required.)

Student Population, Baseline and Target(s) Training SLO Student Population:All 105 students across 4 sections of 7 th grade visual arts classes (names included on the Excel file) Training SLO Baseline: Students scored between 0-30% on the grade 7 visual arts pre-assessment 48 StudentPre-Test ScoreSummative Target Student A10 Student B20 Student C5 Student D0 Student E30 Student F10

Student Population, Baseline, and Target(s) Training SLO Target Approach 1: Set a common growth target. 90% of students, including special populations, will grow by 60 percentage points or more on their summative assessment compared to their pre-test for the standards. (e.g., Student E’s target is 60 more than 30, or 90.) 49 StudentPre-Test ScoreSummative Target Student A1070 Student B2080 Student C565 Student D060 Student E3090 Student F1070

Student Population, Baseline, and Target(s) Training SLO Target Approach 2: Set a growth to mastery target. 85% of students, including special populations, will grow to score 75% or higher on the summative assessment for the selected standards. 50 StudentPre-Test ScoreSummative Target Student A1075 Student B2075 Student C575 Student D075 Student E3075 Student F1075

Student Population, Baseline, and Target(s) Training SLO Target Approach 3: Set differentiated growth targets by student. 85% of students, including special populations, will meet or exceed their individualized target. 51 StudentPre-Test ScoreSummative Target Student A1080 Student B2080 Student C575 Student D070 Student E3085 Student F1080

Student Population, Baseline, and Target(s) 52 Team Activity: For your selected course, identify a sample student population. As a team, decide what type of approach you are using to set growth targets for your SLO. Generate a sample baseline and target for each student in your sample student population.

Discussing District Decisions Decision 4: Establish expectations for scoring SLOs and for determining teacher ratings for the growth component. Begin to think how a district may best engage in establishing HEDI structures. Considerations include how to develop HEDI structures specific to each teaching assignment, how target-setting expectations link to HEDI expectations, how to ensure special populations are not left behind, etc. 53

HEDI This is how different levels of student growth will translate into one of four rating categories: Highly effective, Effective, Developing, and Ineffective Task: Provide specific descriptions of student learning for each rating category. 54

Review of Target Approach 1 – Training SLO Training SLO Target Approach 1: Set a common growth target. 90% of students will grow by 60 percentage points or more on their post-test compared to their pre-test for the standards. (e.g., Student E’s target is 60 more than 30, or 90.) 55 StudentPre-Test ScoreSummative Target Student A1070 Student B2080 Student C565 Student D060 Student E3090 Student F1070

HEDI Approach 1 – Training SLO 56 Training SLO HEDI Approach 1: Set ratings using the percent of students meeting individual targets. 90% of students will grow by 60 percentage points or more on their summative assessment compared to their pre-test for the standards. Highly Effective* (18-20 points) Effective* (9-17 points) Developing* (3-8 points) Ineffective* (0-2 points) % of students grew by 60 points or more on the standards addressed =20 points 97=19 points 96=18 points 90-95% of students grew by 60 points or more on the standards addressed 95=17 points 94=15-16 points 93=13-14 points 92= points 91=10 points 90=9 points 80-89% of students grew by 60 points or more on the standards addressed 89=8 points 87-88=7 points 85-86=6 points 83-84=5 points 81-82=4 points 80=3 points 79% of students or less grew by 60 points or more on the standards addressed 70-79=2 points 60-69=1 point 0-59=0 points *These scoring bands are based on proposed Executive Budget legislation.

Review of Target Approach 2 – Training SLO Training SLO Target Approach 2: Set a growth to mastery target. 85% of students, including special populations, will grow to score 75% or higher on the summative assessment for the selected standards. 57 StudentPre-Test ScoreSummative Target Student A1075 Student B2075 Student C575 Student D075 Student E3075 Student F1075

HEDI Approach 2 – Training SLO 58 Training SLO HEDI Approach 2: Set ratings using the percent of students meeting a collective target (e.g., 80% mastery). 85% of students, including special populations, will grow to score 75% or higher on the summative assessment for the selected standards. Highly Effective* (18-20 points) Effective* (9-17 points) Developing* (3-8 points) Ineffective* (0-2 points) % of students grew to score 75% or higher on the summative assessment rubric measuring the selected standards =20 points 95-97=19 points 91-94=18 points 85% -90% of students grew to score 75% or higher on the summative assessment rubric measuring the selected standards 90=17 points 89=16 points 88=14-15 points 87=12-13 points 86=10-11 points 85=9 points 79% - 84% of students grew to score 75% or higher on the summative assessment rubric measuring the selected standards 84=8 points 83=7 points 82=6 points 81=5 points 80=4 points 79=3 points 78% of students or less grew to score 75% or higher on the summative assessment rubric measuring the selected standards 70-78=2 points 60-69=1 point 0-59=0 points *These scoring bands are based on proposed Executive Budget legislation.

HEDI – Applying 59 Team Activity: For your selected course and target, design a sample HEDI structure.

Discussing District Decisions Decision 5: Determine district-wide processes for setting, reviewing, and assessing SLOs in schools. Given the sample SLO you and your colleagues have developed, review the element selections made and why they were selected. Discuss at what level these decisions should be made for SLOs, and how this work will begin to operate within the district framework. 60

Rationale This describes the reasoning behind the choices regarding learning content, evidence, and target. 61

Rationale – Training SLO Training SLO Rationale: The elements and principles of art are fundamental and core to the visual arts; in this SLO students must skillfully apply them. Analysis, interpretation, and reflection grow deeper thinking processes and communication skills. By coupling the standards for the arts with those of the Common Core, students are sufficiently prepared for future work by reaching this target. 62

Rationale – Training SLO The evidence authentically measures how students progress through key developmental phases. The target requires students to demonstrate significant growth for linchpin aspects of the NYS Arts Standards. Integrating writing with art lessons will be a key instructional approach, along with the use of SIOP strategies for ELLs. My district job-alike colleagues and I will meet bi- weekly to review student progress, discuss successful instructional techniques, and provide differentiation so that each student’s target is met, preparing them for future success. 63

Rationale – Application 64 Team Activity: For your selected course, craft the rationale for the SLO you created today.

Participant SLO Sharing In your teams, review the elements of your SLO, determine the extent to which the elements integrate as a whole, and revise as needed. Chart your sample SLOs on poster paper. Review colleagues’ SLOs around the room. 65

Thank you for your participation! ~ See you soon ~