Presentation is loading. Please wait.

Presentation is loading. Please wait.

B ALTIMORE C ITY P UBLIC S CHOOLS Teacher Evaluation Field Test School Leader Development January 22/23, 2013 1.

Similar presentations


Presentation on theme: "B ALTIMORE C ITY P UBLIC S CHOOLS Teacher Evaluation Field Test School Leader Development January 22/23, 2013 1."— Presentation transcript:

1 B ALTIMORE C ITY P UBLIC S CHOOLS Teacher Evaluation Field Test School Leader Development January 22/23,

2 B ALTIMORE C ITY P UBLIC S CHOOLS Agenda 8:00-8:30 – Registration and Breakfast 8:30-9:20 – Address from Dr. Sonja Brookins Santelises and Jennifer Bell-Ellwanger 9:20-9:40 – Q & A Session with Dr. Sonja Brookins Santelises and Jennifer Bell-Ellwanger 9:40-9:55 – Break 9:55-11:45 – Student Learning Objectives 11:45-12:00 – Q & A session on Student Learning Objectives 12:00-12:45 – Lunch 12:45-1:45 – Value-Added 1:45-2:00 – Q & A on Value-Added 2:00-2:15 – Break 2:15-2:45 – Professional Responsibilities 2:45-3:30 –Student Surveys 3:30-3:45 – Q & A on Student Surveys 3:45-4:00 – Close-Out 2

3 B ALTIMORE C ITY P UBLIC S CHOOLS The Problem… “Current teacher evaluation systems are not providing the information needed to close the achievement gap. Despite 40 years of research pointing to huge differences in student achievement gains across teachers, most school districts…cannot pinpoint what makes a teacher effective, or identify their most and least effective teachers” 3 Working with Teachers to Develop Fair and Reliable Measures of Effective Teaching Bill & Melinda Gates Foundation

4 B ALTIMORE C ITY P UBLIC S CHOOLS The Problem… “The Widget Effect describes the tendency of school districts to assume classroom effectiveness is the same from teacher to teacher. This decades-old fallacy fosters an environment in which teachers cease to be understood as individual professionals…In its denial of individual strengths and weaknesses, it is deeply disrespectful to teachers; in its indifference to instructional effectiveness, it gambles with the lives of students.” 4 The Widget Effect: Our National Failure to Acknowledge and Act on Differences in Teacher Effectiveness The New Teacher Project

5 B ALTIMORE C ITY P UBLIC S CHOOLS The Road Behind: Teacher Evaluation of the Past 5 Infrequent Unfocused Undifferentiated Unhelpful Inconsequential

6 B ALTIMORE C ITY P UBLIC S CHOOLS The Road Ahead: An Evaluation System Grounded in Development “Evaluations should provide all teachers with regular feedback that helps them grow as professionals, no matter how long they have been in the classroom. Evaluations should give schools the information they need to build the strongest possible instructional teams, and help districts hold school leaders accountable for supporting each teacher’s development. Most importantly, they should focus everyone in a school system…on what matters most: keeping every student on track to graduate from high school ready for success in college or a career.” 6 Teacher Evaluation 2.0 The New Teacher Project

7 B ALTIMORE C ITY P UBLIC S CHOOLS The Road Ahead: An Evaluation System Grounded in Student Achievement 7 Improved Teacher Evaluation Based on Multiple Measures Data-Driven Professional Development Strategic Human Capital Decisions Improved Teaching Practices Improved Student Outcomes

8 B ALTIMORE C ITY P UBLIC S CHOOLS Implications for City Schools City Schools is committed to creating a new evaluation process grounded in teacher development that promotes student achievement  Clear, rigorous expectations  Multiple measures  Differentiated ratings  Regular feedback  Significance 8

9 B ALTIMORE C ITY P UBLIC S CHOOLS Implications for City Schools 9 Teachers should be evaluated against clear, rigorous performance expectations based primarily on evidence of student learning. Expectations should reflect excellence in the classroom, not minimally acceptable performance. Clear, Rigorous Expectations No single data point can paint a complete picture of a teacher’s performance Evaluations should incorporate multiple measures for assessing professional practice and student growth, with greater weight afforded to the most accurate measures of student progress. Multiple Measures Teachers should earn a summative rating at the end of each school year that distinguishes performance between levels with meaningful differentiation of performance Teachers should have a clear picture of their current performance. Differentiated Ratings

10 B ALTIMORE C ITY P UBLIC S CHOOLS Implications for City Schools 10 Evaluations should not be limited to a single end-of-year rating, but instead are viewed as an ongoing developmental process. Cultivate a performance-focused culture with frequent teacher observations. Regular conversations with teachers to discuss overall classroom performance and student progress; professional goals and developmental needs; and the support school leaders will provide to meet those needs. Regular Feedback Evaluations should have meaningful implications Teacher effectiveness should play an important role in determining strategic human capital decisions. Ability to identify, develop, and keep talented teachers is one of the most important priorities for schools leaders Significance

11 B ALTIMORE C ITY P UBLIC S CHOOLS Teacher Evaluation Field Test Measures 11 Teacher Evaluation Student GrowthSchool index Observations/ Instructional Framework Student Surveys Professional Responsibilities Student Learning Objectives (SLO) Value- Added Model (VAM) 50%: Multiple measures of student growth 50%: Professional practice City Schools will propose weights after studying the results of the pilot and consulting with our teachers and principals. No one measure can account for more than 35% of a teacher’s evaluation.

12 B ALTIMORE C ITY P UBLIC S CHOOLS Why did we choose these measures? City Schools believes in the following:  Measures should help inform and improve teacher practice  Measures should demonstrate both validity and reliability  Measures should align with teaching best practices and research 12

13 B ALTIMORE C ITY P UBLIC S CHOOLS Why did we choose these measures? Research from the MET Project 13 What is the MET Project? The MET project, funded by the Gates Foundation, is testing five different measures of teaching effectiveness: 1. Student achievement gains on state standardized tests and supplemental tests 2. Classroom observations and teacher reflections 3. Teachers' pedagogical content knowledge 4. Student perceptions of the classroom instructional environment MET stands for Measures of Effective Teaching, and the project aims to help teachers and schools understand what great teaching looks like. The goal of the MET project is to identify which multiple measures give the best and most accurate information about how well a teacher helps his or her students learn – and how these measures should be used together to see the whole picture of a teacher's effectiveness.

14 B ALTIMORE C ITY P UBLIC S CHOOLS Two school years: and >100,000 students Grades 4-8: ELA and Math High School: ELA I, Algebra I and Biology The MET Study 14

15 B ALTIMORE C ITY P UBLIC S CHOOLS Three Key Take-Aways 1. High-quality classroom observations will require clear standards, certified raters, and multiple observations per teacher. 2. Combining three approaches (classroom observations, student feedback, and value-added student achievement gains) capitalizes on their strengths and offsets their weaknesses. 3. Combining new approaches to measuring effective teaching – while not perfect – significantly outperforms traditional measures. Providing better evidence should lead to better decisions. 15

16 B ALTIMORE C ITY P UBLIC S CHOOLS Why did we choose these measures? 16 ComponentRationale: Why this measure?Rationale: Why this tool? Student growth: VAM and SLOs Under our theory of action, student achievement is the ultimate end of our evaluation work. Student growth is our business as educators. RTTT and the Ed Reform Act require measures of student growth as a significant part of the evaluation. VAM provides longitudinal achievement data and controls for factors outside the teacher’s control. VAM is cost- effective because it can be used to measure student growth for many teachers. SLOs allow us to measure student growth in non-tested content areas, and they support instructional best practices. Instructional Framework Professional development is critical to any evaluation system. The Framework provides teachers and school leaders with guidance on how to improve professional practices. The Framework defines highly effective teaching practices in Baltimore City, and provides teachers with specific, actionable feedback. Professional Responsibilities Checklist Professional responsibilities are included of most teaching frameworks. Teachers should model professionalism expectations for our students. The Checklist evaluates teachers on non-instructional functions. Checklist reflects key district priorities, such as teacher attendance and arrival, compliance with policies and testing integrity. Student SurveysStudents recognize effective teaching when they experience it. Students are our customers, and have more direct contact with teachers than anyone else. The Survey contains items that are aligned to Framework indicators and developmentally appropriate for students. Survey data reinforces feedback that teacher receive from classroom observations. School IndexTeaching is a team sport. All staff in a school are responsible for certain student outcomes. School indexes consider indicators such as attendance or a positive school climate, which the entire school can contribute to improving.

17 B ALTIMORE C ITY P UBLIC S CHOOLS Teacher Involvement 17 How are teachers providing feedback on measures?  Educator Support and Evaluation Committee (ESEC) – 4 principals, 16 teachers, and an ED How can teachers learn more about the evaluation field test?  s from City Schools  Webinars at ILT and faculty meetings  City Schools’ website, under “About Us” and click on “Effectiveness”  BTU/City Schools information sessions beginning February 5

18 B ALTIMORE C ITY P UBLIC S CHOOLS Fall 2012Winter 2013 Spring 2013 Summer 2013 What Will Teachers Experience in ? 18 All teachers in grades 3-12 have at least one grade/ course/section complete a student survey. Teachers will validate their course rosters. All teachers receive updates on VAM as well as other measures in the teacher evaluation field test Summer 2013: All teachers receive a composite rating SLOs will be rolled out gradually to all schools beginning Jan Professional responsibilities component will be outlined for teachers in Jan During SY : Teachers will experience two formal observations using PBES with Framework 2.0 as the guiding document. Launch Field Test via blast from Dr. Alonso, and joint message with Marietta English.

19 B ALTIMORE C ITY P UBLIC S CHOOLS Questions? 19

20 B ALTIMORE C ITY P UBLIC S CHOOLS Like Me? Type of Leader? Product of City Schools? Years in City Schools? Years in Leadership? 20

21 B ALTIMORE C ITY P UBLIC S CHOOLS Teacher Evaluation Field Test Components 21 Teacher Evaluation Student GrowthSchool index Observations/ Instructional Framework Student Surveys Professional Responsibilities Student Learning Objectives (SLO) Value- Added Model (VAM) 50%: Multiple measures of student growth 50%: Professional practice City Schools will propose weights after studying the results of the pilot and consulting with our teachers and principals. No one measure can account for more than 35% of a teacher’s evaluation.

22 B ALTIMORE C ITY P UBLIC S CHOOLS Session Objectives Understand the major components of a City Schools Student Learning Objective (SLO), Develop an understanding of the rationale behind using SLOs as part of the teacher evaluation system, Discuss the potential challenges and learning opportunities that SLOs provide, Consider the impact that school leaders and teacher- leaders may have on the successful roll-out of SLOs, Begin to consider potential candidates as SLO Ambassadors 22

23 B ALTIMORE C ITY P UBLIC S CHOOLS City Schools: the Numbers  Approximately ~35-45 % of teachers* fall into the “tested” category  In other words, 35-45% of our teachers teach subjects assessed by the MSA, HSA, or Stanford-10  The other 55-65% of our teachers fall into the “non-tested” category  These teachers do not teach subjects directly assessed by the MSA, HSA, or Stanford-10 * Estimates based on SMS data, Fall

24 B ALTIMORE C ITY P UBLIC S CHOOLS Think-Pair-Share Silently consider your response to the following question: What tools do we currently have to analyze student growth to help our teachers in the non-tested grades and subjects (NTGS)? Turn to the person beside you and discuss your response Share what you heard your discussion partner say 24

25 B ALTIMORE C ITY P UBLIC S CHOOLS Independent Reading Read the first four pages of the SLO Brief provided in your folder (10 min) While reading consider the following questions:  Why are SLOs proposed to be a part of the teacher evaluation system?  In what ways do SLOs connect to other district initiatives (ie SPPs, etc)?  Where do you see yourself having the most impact on SLOs? 25

26 B ALTIMORE C ITY P UBLIC S CHOOLS The Research “In Denver, students whose teachers crafted high quality SLOs showed more than a year’s worth of gain… during each year of the study at all three school levels.” “In Charlotte-Mecklenburg, during the first phase of implementation we found another strong correlation between the targeted objective and student achievement….” Source: Community Training and Assistance Center,

27 B ALTIMORE C ITY P UBLIC S CHOOLS Why SLOs? Rationale: Of the measures of student growth we studied for non-tested areas, SLOs had the most potential for improving practice  SLOs reinforce good teaching practices  What all of our best teachers are already doing anyway  Are not compliance-based  Data-driven instruction  Provides an anchor and theme for collaborative planning 27

28 B ALTIMORE C ITY P UBLIC S CHOOLS 28 “If properly implemented, student learning objectives help teachers bring more science to their art, strengthen instructional support to the classrooms, and improve the quality of the outcome.” William J. Slotnik Founder and Executive Director Community Training and Assistance Center

29 B ALTIMORE C ITY P UBLIC S CHOOLS SLO Cycle REVIEW STUDENT DATA Individual teachers or teams of teachers review their student data before the school year begins (or pre-test data after). (August-September) DRAFT SLOS Individual teachers or teams of teachers draft SLOs. (September-October) SLO APPROVAL Evaluators review and approve proposed SLOs (or request revisions). (October-November) MID-YEAR REVIEW Teachers and/or evaluators may review teacher progress on SLO targets mid-year. Changes in assignment or class composition may warrant target adjustment. (February) SLO SCORING Evaluator reviews and scores teachers’/teams of teachers’ progress on SLOs. (May-June) SUMMATIVE SCORING SLO results are included among measures of student growth in summative ratings of teachers’ performance. (June) REFLECTION Teachers/teams of teachers and evaluator discuss progress and implications for next year’s SLO(s) (June, Summer)

30 B ALTIMORE C ITY P UBLIC S CHOOLS Exploring an SLO At your table you will be given a model SLO Review the document silently for 5 minutes Respond to the four questions on the Four Square document provided Take 10 minutes to discuss your responses with your table Share out your group’s responses 30

31 B ALTIMORE C ITY P UBLIC S CHOOLS What is an SLO? Learning goal for students Specific to a group of students for that year Measurable Collaborative: set in consultation with colleagues and evaluator(s) Organic to good teaching practice Popular: Many states and districts are building systems of educator effectiveness that include SLOs as one of multiple measures in teacher and principal evaluations, including:  Georgia, Indiana, Louisiana, New York, Rhode Island  Austin, Charlotte-Mecklenburg, Denver, Houston, New Haven 31

32 B ALTIMORE C ITY P UBLIC S CHOOLS Which Teacher Will Complete SLOs? All teachers except those with students that take the following assessments:  Stanford 10, grades 2  Teachers with students that take Stanford 10 in grade 1 will write SLOs  MSA Reading and Math, grades 3-8  Courses that trigger HSA Algebra, Biology, English, and Government in grades

33 B ALTIMORE C ITY P UBLIC S CHOOLS Fall 2012Winter 2013 What Has Already Happened with SLOs? 33 SLO bank and guidebook are created by January Principals receive training in January, with materials they will take back to their buildings so that SLOs can be rolled out to all schools for SLO field test in February SLO WRITING WORKSHOP – Content Leaders and non-tested subject teachers Progress to date: -September /video from Dr. Alonso and Marietta English announcing field test -SLO 101 (2 sessions) for central office personnel -SLO Tools Development Workshop (November ) -On-going policy decisions by City Schools leadership team

34 B ALTIMORE C ITY P UBLIC S CHOOLS JanFebMarchAprilMayJuneJulyAug SLO Implementation for Field Test 34 Feb-March: Ambassadors attend half- day PD on SLO 101; content-based workgroups trained on SLO writing (Dates: Feb. 12 th, March 19 th ) June-July: OAA collects and analyzes SLO data and reports to MSDE Following Principals’ training on 1/22-1/23: Principals select 2 SLO ambassadors at their schools April: Ambassadors draft SLOs in work groups; principals trained on SLO approval (April 2 nd ); principals approve ambassadors’ SLOs June: SLOs scored by district staff. August/September: All non-tested teachers trained on SLO 101 and SLO Writing

35 B ALTIMORE C ITY P UBLIC S CHOOLS What role will you play in SLO Development? Nominate 2 SLO Ambassadors  Send 2 nominations to by January 31  Assist with coverage for these teachers for 3-5 hours a month Attend SLO approval training on April 2 nd Approve SLOs that have been written by Ambassadors–ensuring “ambitious, yet attainable” 35

36 B ALTIMORE C ITY P UBLIC S CHOOLS SLO Ambassadors: Role Description There will be 2 SLO Ambassadors at each school. SLO Ambassadors will:  Attend monthly trainings  Submit draft SLOs to principals for approval  In SY13-14, become school-based SLO experts and provide SLO support in their buildings 36

37 B ALTIMORE C ITY P UBLIC S CHOOLS SLO Ambassadors: Selection Criteria SLO Ambassadors Should:  Teach in a non-tested content area; i.e. subjects/grades not covered by MSA/HSA exams  Demonstrate strength in instructional practices during classroom observations using the Framework  Be able to commit 3-5 hours per month to attend trainings, workgroup meetings, and complete outside work, as needed  Plan to return to same school for SY13-14, and facilitate and lead SLO trainings to colleagues  Candidates may be Model Teachers 37

38 B ALTIMORE C ITY P UBLIC S CHOOLS SLO EXAMPLE: Basic Information Teacher: Nick GreerGrade Level: 7 School:Alice Pinderhughes High # of Students Addressed by SLO: 110 Subject: 7 th Grade Life Science Interval of Instruction: X Full Year ☐ Semester ☐ Other: 38

39 B ALTIMORE C ITY P UBLIC S CHOOLS SLO Writing: Assessment Assessment should:  Answer the question: “How will I know if students have learned the key content?” – it provides evidence  Be aligned to standard(s) targeted  Meet quality criteria that will be established by City Schools For teacher-created SLOs, describes the assessment Assessments can be standardized, but they also can be performance tasks or portfolios that are judged against a rubric 39

40 B ALTIMORE C ITY P UBLIC S CHOOLS SLO Writing: Standards Addressed by SLO Lists specific standards that will have been met/mastered if the SLO is achieved Address critical content, skills or knowledge necessary for advancement to future coursework  That is, if students do not master these standards, they will not be able to progress to the next level  No need to list every single standard 40

41 B ALTIMORE C ITY P UBLIC S CHOOLS SLO Writing: Baseline Data Baseline data can be collected from:  Pre-assessments  In-class assignments  Observation Include any additional student data or background information used in setting the target (e.g. ESOL status, SWD incidence) 41

42 B ALTIMORE C ITY P UBLIC S CHOOLS SLO Writing: Student Learning Objective States the specific target for student achievement by the end of the instructional period Target should be ambitious, yet achievable, given the students’ baseline data 42

43 B ALTIMORE C ITY P UBLIC S CHOOLS SLO Hazards These common errors make an SLO unscorable: THE DOUBLE TARGET SLO: “My students will get to [this level] by January and [this level] by May” Or “My students’ average score will be 84 and no score will be lower than 70” THE VAGUE SLO: “My students will all be able to write an excellent explanatory essay by May.” THE NARROW SLO: “My students will be able to describe three ways that the 1787 Constitutional Convention influences our country today.” 43

44 B ALTIMORE C ITY P UBLIC S CHOOLS SLO Writing: Encouraging Powerful Teacher Practices 44

45 B ALTIMORE C ITY P UBLIC S CHOOLS Turn and Talk How can these sections impact collaboration in your school and across the district? What practices will you plan to use here to encourage teachers to collaborate? 45

46 B ALTIMORE C ITY P UBLIC S CHOOLS Scoring Plan 46 Scoring Plan: Insufficient Attainment of Target (1 point): The teacher has demonstrated an insufficient impact on student learning by falling far short of the target. Partial Attainment of Target (2 points): The teacher has demonstrated some impact on student learning, but did not meet the target. Full Attainment of Target (3 points): The teacher has demonstrated a considerable impact on student learning by meeting the target. Exceptional Attainment of Target (4 points): The teacher has demonstrated an outstanding impact on student learning by surpassing the target by a meaningful margin. Achievement Range:

47 B ALTIMORE C ITY P UBLIC S CHOOLS Debrief and Questions How would you explain what an SLO is to your teachers? Why are we using them? What systems will you need in place to ensure that your teachers are successful with SLOs? How can the Ambassadors best support quality SLO work? What outstanding questions do you still have regarding SLOs? 47

48 B ALTIMORE C ITY P UBLIC S CHOOLS Next Steps SLO 101 for School Leaders (Today) Select 2 SLO ambassadors (By Jan 31) Ambassadors attend half-day PD on SLO 101; content-based workgroups trained on SLO writing (Dates: Feb. 12 th, March 19 th ) Ambassadors draft SLOs in work groups; principals trained on SLO approval (April 2 nd ); SLOs scored by district staff (June) OAA collects and analyzes SLO data and reports to MSDE (June-July) All non-tested teachers trained on SLO 101 and SLO Writing (August/September) 48

49 B ALTIMORE C ITY P UBLIC S CHOOLS Support and Resources For more information on Student Learning Objectives, please contact us at 49

50 B ALTIMORE C ITY P UBLIC S CHOOLS LUNCH! School Leader Development January 22/23,

51 B ALTIMORE C ITY P UBLIC S CHOOLS Teacher Evaluation Field Test Components 51 Teacher Evaluation Student GrowthSchool index Observations/ Instructional Framework Student Surveys Professional Responsibilities Student Learning Objectives (SLO) Value- Added Model (VAM) 50%: Multiple measures of student growth 50%: Professional practice City Schools will propose weights after studying the results of the pilot and consulting with our teachers and principals. No one measure can account for more than 35% of a teacher’s evaluation.

52 B ALTIMORE C ITY P UBLIC S CHOOLS Session Objectives Develop an understanding of the rationale behind using value-added as part of the teacher evaluation system Gain a deeper understanding of how value-added models work in general and the details of City School’s value-added model Discuss the potential challenges and learning opportunities that value-added models (VAM) provide 52

53 B ALTIMORE C ITY P UBLIC S CHOOLS Why Use Value-Added Analysis? Teachers can and should contribute to student learning from August to June. This is true whether the student comes in above grade, at grade or below grade. Value added analysis shows how well teachers do with all students, regardless of their initial starting points – unlike simple proficiency scores. 53

54 B ALTIMORE C ITY P UBLIC S CHOOLS VAM Research: Teachers Matter  Value-added measures of teacher effectiveness are positively correlated with principals’ subjective assessments of teachers. 1  VAM measures are useful for differentiating between the highest and lowest performing teachers. 2  Teachers with high value-added scores make significant contributions to their students’ learning, regardless of the student population Jacob and Lefgren (2005); 2 Rivkin,Hanushek, & Kain (2005); 3 Kane et. al (2008)

55 B ALTIMORE C ITY P UBLIC S CHOOLS VAM Research: Teachers Matter 55 Researchers have found evidence, using value-added models, showing that:  Highly effective teachers can have an immediate impact on student achievement.  Students assigned to highly effective teachers are more likely to go to college, earn higher incomes and less likely to become teenage parents.  Having a highly effective teacher for one year increases a child’s cumulative lifetime income by $50K. Source: Chetty, Raj et al. The Long Term Impacts of Teachers: Teacher Value Added and Student Outcomes in Adulthood. December Available at

56 B ALTIMORE C ITY P UBLIC S CHOOLS Understanding Value-Added The following video will walk us through the difference between the following ways of looking at student test scores  Achievement Model  Simple Growth or Gain Model  Value-Added Model If you would like to use this video with your teachers, it can be found at 56

57 B ALTIMORE C ITY P UBLIC S CHOOLS How does City Schools’ VAM work? Value-Added is defined as the teacher’s contribution to student growth. The model calculates a predicted test score for a student in a given grade and subject. The predicted score is based on the student’s prior academic achievement (and other factors). Then, we compare the student’s predicted score to the actual score in the grade and subject. The difference between the predicted score and the actual score is called the teacher’s value-added score. 57

58 B ALTIMORE C ITY P UBLIC S CHOOLS How does City Schools’ VAM work? Graphic Representation Value Added Student Predicted Gain Student Value Added Student Actual Score Student Predicted Score = Test Score Baseline (Previous Year’s Score) Current - 58 Score

59 B ALTIMORE C ITY P UBLIC S CHOOLS How does City Schools’ VAM work? Aggregating Teacher VAM Scores Test Score Baseline (Previous Year’s Score) Current Student A’s VAM Score 59

60 B ALTIMORE C ITY P UBLIC S CHOOLS How does City Schools’ VAM work? Aggregating Teacher VAM Scores Test Score Baseline (Previous Year’s Score) Current Student B’s VAM Score Student A’s VAM Score 60

61 B ALTIMORE C ITY P UBLIC S CHOOLS How does City Schools’ VAM work? Aggregating Teacher VAM Scores = Test Score Baseline (Previous Year’s Score) Teacher Value-Added Score Student A’s VAM Score Student B’s VAM Score + + … Current Student B’s VAM Score Student A’s VAM Score 61

62 B ALTIMORE C ITY P UBLIC S CHOOLS Explaining Value-Added Take 2 minutes to think through the following question:  How would you explain value-added to your teachers? Turn to a partner and try out this explanation (both people try it!) 62

63 B ALTIMORE C ITY P UBLIC S CHOOLS Which Teachers Receive VAM Scores? Teachers with students that take the following assessments:  Stanford 10, grades 2  MSA Reading and Math, grades 3-8  Courses that trigger HSA Algebra, Biology, English, and Government in grades

64 B ALTIMORE C ITY P UBLIC S CHOOLS Current Control Variables 64 Student-Level ControlsSchool-Level Controls 1.Prior Achievement (2 years of prior scores)1.Percent of students in school receiving free/reduced price lunch 2.Student’s prior year number of days absent2.Percent of students in the school classified as special education students 3.ELL Status3.Percent of students in school out of age level 4.Mobility (number of transitions)4.Average number of special education service hours per student 5.Difference from modal age in grade 6.Repeater status (i.e. did student repeat a prior grade?) 7.Skipper status (i.e. did student skip a prior grade?)

65 B ALTIMORE C ITY P UBLIC S CHOOLS Roster Validation Roster validation allows us to assign the correct students to teachers Without validation, it is possible for students that are incorrectly on a teacher’s roster to count towards his or her value-added or SLO score Teacher verification is planned for late April, with principal review and approval scheduled for early May 65

66 B ALTIMORE C ITY P UBLIC S CHOOLS Support and Resources For more information on value-added, please contact us at 66

67 B ALTIMORE C ITY P UBLIC S CHOOLS Teacher Evaluation Field Test Components 67 Teacher Evaluation Student GrowthSchool index Observations/ Instructional Framework Student Surveys Professional Responsibilities Student Learning Objectives (SLO) Value- Added Model (VAM) 50%: Multiple measures of student growth 50%: Professional practice City Schools will propose weights after studying the results of the pilot and consulting with our teachers and principals. No one measure can account for more than 35% of a teacher’s evaluation.

68 B ALTIMORE C ITY P UBLIC S CHOOLS Session Objectives Understand City Schools’ Professional Responsibilities Checklist and how it fits in with the teacher evaluation field test Discuss ways to capture and record Professional Responsibilities data during the field test 68

69 B ALTIMORE C ITY P UBLIC S CHOOLS What are “professional responsibilities”? Definition: Professional Responsibilities are the core, non-instructional job functions of teachers.  Examples: showing up to work, punctuality, turning in grades on time, attending staff meetings, etc.  Professional responsibilities are not unique to teaching. These expectations are consistent across all, or most, professions. 69

70 B ALTIMORE C ITY P UBLIC S CHOOLS Why measure professional responsibilities? Three-Pronged Rationale: 1. Professional responsibilities are threshold skills to effective teaching. 2. Professional responsibilities are included in most other teaching frameworks (see e.g. the Danielson Framework, Domain 4 of PBES). 3. City Schools’ staff should model the same expectations we have of our students. 70

71 B ALTIMORE C ITY P UBLIC S CHOOLS What does the checklist look like? To create our Checklist, we reviewed professionalism measures in other school districts, generated a list of possible criteria (including PBES), and shared drafts with teachers for feedback. The Checklist is comprised of four categories: 1. Attendance 2. Arrival 3. Compliance with District and School Policies 4. Testing Integrity 71

72 B ALTIMORE C ITY P UBLIC S CHOOLS Category 1: Attendance 72 CategoryMeets ExpectationsDoes Not Meet Expectations AttendanceTeacher has zero or 1 unexcused absence. The term “unexcused absence” should be interpreted in light of the BTU contract and City Schools’ Attendance Reliability and Analysis Program. In other words, an unexcused absence is an absence that is not eligible for leave as specified in Art. XV in the BTU contract OR an absence (or occasion) in excess of seven previous periods of absence. Teacher has 2 or more unexcused absences.

73 B ALTIMORE C ITY P UBLIC S CHOOLS Category 2: Arrival 73 CategoryMeets ExpectationsDoes Not Meet Expectations ArrivalTeacher has no more than 2 unexcused late arrivals. The term “unexcused late arrival” should be interpreted in light of the BTU contact and City Schools’ Attendance Reliability and Analysis Program. In other words, an unexcused late arrival occurs when a teacher is not at his/her duty post prior to the start of the work day, prior notification notwithstanding. At the discretion of the supervisor, a teacher may meet expectations if, after two unexcused late arrivals and a discussion with his/her supervisor, he/she improves his/her punctuality over the rest of the school year. Teacher has more than 2 unexcused late arrivals, AND fails to make improve his/her punctuality over the rest of the school year.

74 B ALTIMORE C ITY P UBLIC S CHOOLS Category 3: Compliance with District and School Policies 74 CategoryMeets ExpectationsDoes Not Meet Expectations Compliance with District and School Policies Teacher meets deadlines for submitting progress reports, grades and attendance data, AND Teacher always or usually follows all other school and district policies and procedures (e.g. administer benchmarks, take daily attendance, etc.), AND Teacher attends all mandatory meetings, conferences and trainings during the regular school day. At the discretion of supervisor, a teacher may meet expectations if, after failure to follow district or school policies and a discussion with his/her supervisor, he/she improves his/her performance over the rest of the school year. Teacher demonstrates a pattern of missing deadlines for submitting progress reports, grades and attendance data, OR Teacher demonstrates a pattern of failing to follow school and district policies and procedures, OR Teacher demonstrates a pattern of missing mandatory meetings, conferences or trainings during the school day, AND Does not improve his/her compliance with school or district polices over the rest of the school year.

75 B ALTIMORE C ITY P UBLIC S CHOOLS Category 4: Testing Integrity 75 CategoryMeets ExpectationsDoes Not Meet Expectations Testing Integrity Teacher strictly complies with all district policies regarding testing security, AND Teacher promptly discloses to school or district leadership any incidents of cheating or testing impropriety. Teacher does not strictly comply with all district policies regarding testing security, OR Teacher fails to disclose incidents of cheating or testing impropriety to school or district leadership.

76 B ALTIMORE C ITY P UBLIC S CHOOLS JanFebMarchAprilMayJuneJulyAug Professional Responsibilities Checklist Implementation 76 June: OAA will provide school leaders with a template for reporting Checklist data. Data templates must be returned to OAA by June 21, 2012 Following Principals’ training on 1/22-1/23: Share Checklist with your staff at an upcoming faculty meeting. Identify any school specific priorities in the compliance category. Now through June: Track data using the Checklist Tracking Sheet. Update teachers on their performance, especially if they are at risk of not meeting expectations in a category.

77 B ALTIMORE C ITY P UBLIC S CHOOLS Discussion Are we setting the right targets for meeting expectations? What best practices have you developed at your school for tracking this kind of information? How can OAA and other central office staff support you in tracking and reporting this data for the field test? 77

78 B ALTIMORE C ITY P UBLIC S CHOOLS Support and Resources For more information on the Professional Responsibilities Checklist, please contact us at 78

79 B ALTIMORE C ITY P UBLIC S CHOOLS Teacher Evaluation Field Test Components 79 Teacher Evaluation Student GrowthSchool index Observations/ Instructional Framework Student Surveys Professional Responsibilities Student Learning Objectives (SLO) Value- Added Model (VAM) 50%: Multiple measures of student growth 50%: Professional practice City Schools will propose weights after studying the results of the pilot and consulting with our teachers and principals. No one measure can account for more than 35% of a teacher’s evaluation.

80 B ALTIMORE C ITY P UBLIC S CHOOLS Session Objectives Develop an understanding of the rationale behind using student surveys as part of the teacher evaluation system Discuss the development of the survey and timeline for implementation of student surveys in the field test Obtain feedback on the proposed student survey items Gain familiarity with interpretation of student survey teacher reports 80

81 B ALTIMORE C ITY P UBLIC S CHOOLS Students have the most contact with teachers and are the direct consumers of a teacher’s service – they are our ultimate customer Multiple measures provide more information and a more robust definition of teacher effectiveness Student surveys provide data that can inform teacher practice Why Student Surveys? 81

82 B ALTIMORE C ITY P UBLIC S CHOOLS Review of Research on Student Surveys Teachers rated higher by students in instructional effectiveness align with students achieving at higher levels in that teacher’s class (MET Project, 2010a; Balch, 2012; Crow, 2011; Kyriakides, 2005). Student ratings are a valid and reliable data source (MET Project, 2010a; Peterson et al., 2000; Worrell & Kuterbach, 2001). Elementary and secondary students are as reliable as older, adult raters in rating teaching behaviors (Follman, 1992, 1995; Worrell & Kuterbach, 2001). 82

83 B ALTIMORE C ITY P UBLIC S CHOOLS When student surveys are combined with rubrics, the difference between the bottom and top teachers increases from 2.6 month to 4.8 months of learning MET Project findings 83

84 B ALTIMORE C ITY P UBLIC S CHOOLS Aligning Survey Items 84 How we aligned the survey items Start with appropriate Instructional Framework TEACH behavior descriptions for Highly Effective and EffectiveInstructional Framework TEACH behavior descriptions Match to pre-validated items Write new items if necessary

85 B ALTIMORE C ITY P UBLIC S CHOOLS What Items Are on the Survey? In your folder is a copy of the student survey that will be field tested Take 5-7 minutes to go through the items individually and then discuss with others at your table  High School Principals – Items 1-17  Middle School Principals – Items  Elementary School Principals – items Guiding questions for review:  What improvements could you make on any of the items?  What items will your teachers be concerned with? 85

86 B ALTIMORE C ITY P UBLIC S CHOOLS Screening procedures were used to identify and eliminate invalid survey responses  First procedure identifies answers that have a minimum difference from a teacher's overall average for a question  Student response to the question “I was being honest when taking this survey” 1  Surveys with answers of either Disagree or Strongly Disagree were eliminated. Screening procedures 1 Reniscow et al., 2001; GAO,

87 B ALTIMORE C ITY P UBLIC S CHOOLS JanFebMarchAprilMayJuneJulyAug Student Survey Implementation 87 March: Student surveys administered in schools from March 1 st to March 22 nd (Administered at the same time as student climate surveys) 1 June-July: OAA analyzes field test data and reports to MSDE Following Principals’ training on 1/22-1/23: Test coordinators will receive information on survey administration Late Spring: Teacher will receive individual student feedback reports 1 PARENT AND TEACHER CLIMATE SURVEYS WINDOW IS FEBRUARY 21 ST TO MARCH 22 ND

88 B ALTIMORE C ITY P UBLIC S CHOOLS Interpreting a Student Survey Report Take 2-3 minutes to think through the following questions:  How do the category scores of this teacher compare to the school and district average?  What are some strategies that you might tell this teacher for each of the ‘Areas for Improvement’?  How else might you be able to use the information from this report to assist in the professional development of a teacher? With a partner, discuss your answers to these questions for 4-5 minutes 88

89 B ALTIMORE C ITY P UBLIC S CHOOLS Which Survey? School (Climate) Surveys – Students  Asks about the climate of the whole school School (Climate) Surveys – Parents School (Climate) Surveys – Teachers Student Survey of Teacher Practice  Asks about individual teachers 89

90 B ALTIMORE C ITY P UBLIC S CHOOLS What is the Role of the Principal? Provide feedback on student survey questions (today) Assist with survey administration (will occur at the same time as school survey for students)  Administration should take place in the classes of the teacher being evaluated  Teachers should switch classrooms  Need to survey the same class periods for high school/middle school (i.e. period 2 and 3), only one survey per teacher in elementary school  Further details provided to test coordinators Jan 29/30 th Assist teachers with interpretation of reports 90

91 B ALTIMORE C ITY P UBLIC S CHOOLS Support and Resources For more information on student surveys, please contact us at 91

92 B ALTIMORE C ITY P UBLIC S CHOOLS Close Out Feedback 92


Download ppt "B ALTIMORE C ITY P UBLIC S CHOOLS Teacher Evaluation Field Test School Leader Development January 22/23, 2013 1."

Similar presentations


Ads by Google