We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byAlissa Potter
Modified over 2 years ago
Reversing The Widget Effect Teacher Union Reform Network I November 9 -10
2 © The New Teacher Project 2009 Agenda About The New Teacher Project Studying Teacher Evaluation The Current Problem The Current Opportunity Preliminary Data About Peer Review Programs How to Build a Better Evaluation System Appendix
3 © The New Teacher Project 2009 The New Teacher Project (TNTP) helps school districts and states fulfill the promise of public education by ensuring that all students— especially those from high-need communities—get excellent teachers. National nonprofit, founded by teachers in 1997 Partners with school districts, state education agencies, and charter schools Targets acute teacher quality challenges Delivers a range of customized services and solutions on a fee-for-service basis Approx. 200 employees, most embedded in school district offices; majority are former teachers Past and present clients include: Districts: Baltimore, Chicago, Denver, Memphis, New Orleans, New York, Oakland, Philadelphia, San Antonio, Washington, DC States: Alaska, Louisiana, Texas, Virginia
4 © The New Teacher Project 2009 Agenda About The New Teacher Project Studying Teacher Evaluation The Current Problem The Current Opportunity Preliminary Data About Peer Review Programs How to Build a Better Evaluation System Appendix
5 © The New Teacher Project 2009 Study Scope In Fall 2008 through Spring 2009, TNTP partnered with 12 school districts in four states to analyze each district’s evaluation, tenure, remediation and dismissal policies and practices as well as each state’s teacher performance management policies. Represented: Urban, suburban and rural districts Collective bargaining and non-collective bargaining states Peer review and traditional evaluation approaches
6 © The New Teacher Project 2009 Data Collection ArkansasColoradoIllinoisOhioTOTAL Districts Studied Teachers Surveyed 2,1962,4287,4823,07015,176 Principals Surveyed ,281 Former Teachers Surveyed N/A Interviews Conducted* Teacher Evaluation Records Collected 1,5885,85528,1477,04042,630 *Interviews conducted with some combination of teachers, principals, local teacher union leaders, local principal professional organization leaders, superintendent, district HR staff, legal counsel, and school board members.
7 © The New Teacher Project 2009 Advisory Panels Membership One panel per state Approximately 80 stakeholders total Advisory Panel members include: oDistrict leaders and HR staff oLocal and state union leadership oSchool board members oState policy makers oEducation advocates oFoundation representatives Three formal meetings, 2-3 hours long. TNTP’s research benefited from the extensive input and feedback of Advisory Panels in each of the four states studied. Role Share information and perspective on state/district context. Suggest issues for additional exploration and sources of information. Comment on study methodology and data collection methods and instruments. React to preliminary findings and recommendations. Craft brief responses to the study for publication in unedited form.
8 © The New Teacher Project 2009 Agenda About The New Teacher Project Studying Teacher Evaluation The Current Problem The Current Opportunity Preliminary Data About Peer Review Programs How to Build a Better Evaluation System Appendix
9 © The New Teacher Project 2009 The Widget Effect “When it comes to measuring instructional performance, current policies and systems overlook significant differences between teachers. There is little or no differentiation of excellent teaching from good, good from fair, or fair from poor. This is the Widget Effect: a tendency to treat all teachers as roughly interchangeable, even when their teaching is quite variable. Consequently, teachers are not developed as professionals with individual strengths and capabilities, and poor performance is rarely identified or addressed.” The New Teacher Project, 2009
10 © The New Teacher Project 2009 Teacher Evaluation: Where We Codify The Widget Effect Treating teachers as interchangeable parts All teachers are rated “good” or “great.” Although teachers and principals report that poor performance is common, less than 1 percent of teachers are identified as “unsatisfactory” on performance evaluations. Excellence goes unrecognized. When excellent ratings are the norm, truly exceptional teachers cannot be formally identified. Nor can they be compensated, promoted or retained. Professional development is inadequate. Almost 3 in 4 teachers did not receive any specific feedback on improving their performance in their last evaluation. Novice teachers are neglected. Low expectations for beginning teachers translate into benign neglect in the classroom and a toothless tenure process. Poor performance goes unaddressed. Half of the 12 districts studied have not dismissed a single non- probationary teacher for poor performance in the past five years. None dismisses more than a few each year.
11 © The New Teacher Project 2009 When multiple ratings are available, teachers tend to be assigned the highest ratings and are very rarely assigned poor ratings.
12 © The New Teacher Project 2009 In districts that use binary “Satisfactory/Unsatisfactory” rating systems, the “Unsatisfactory” rating is almost never used. Evaluation Ratings for Tenured Teachers in Districts with Binary Rating Systems
13 © The New Teacher Project 2009 These data often stand in sharp relief against current levels of student achievement, as this example from Denver illustrates. Frequency of Unsatisfactory Ratings in Denver Public Schools that Did Not Meet AYP In Denver schools that did not make adequate yearly progress (AYP) for three consecutive years ( to ), more than 98 percent of tenured teachers received the highest rating —satisfactory.
14 © The New Teacher Project 2009 Teachers say districts aren’t doing enough to recognize and retain top performers as measured by their impact on student learning… 59% of teachers report their district is not doing enough to identify, recognize, compensate, promote and retain the most effective teachers as measured by their impact on student learning. “All the good quality teachers leave the district after just a few years. They need more incentive to stay.” “Some sort of recognition or praise would be nice. Those doing a good or great job are never told so.” “If you pay the shining stars the same as the slackers, you will dim the shining stars and reinforce the sloth of the slackers.” “I, and others, work hard because we have a conscience, but I don't think [the district] sees us as any different than the lower performing teachers. Teachers who work hard receive very little praise or notice.” TNTP survey of 7,318 teachers across four sites conducted May 2008 to April 2009
15 © The New Teacher Project 2009 Percent of teachers who had development areas identified on their most recent evaluation. Tenured/non- probationary teachers Probationary teachers Most likely (Denver) 32%55% Average 22%37% Least likely (Springdale) 2%4% Source 1 : TNTP survey of 15,176 teachers across 12 sites conducted May 2008 to April 2009 Source 2 : TNTP survey of 1,863 Denver Public School teachers conducted November to December 2008 …or to ensure that evaluation practices and systems provide them with meaningful feedback. of Denver teachers who had a development area identified on their most recent evaluation “do not know” which performance standard they failed to meet. 39%
16 © The New Teacher Project 2009 And despite the fact teachers and principals agree that poor instruction is pervasive… Source: TNTP survey of 7,318 teachers and 751 administrators across 4 sites conducted February to April 2009; teacher evaluation data provided by each district. “Are there tenured/non-probationary teachers in your school who deliver poor instruction?” (Percent responding “Yes”) 0%0.4%n/a0% Percent of All Ratings that Indicated “Unsatisfactory” Performance
17 © The New Teacher Project 2009 …dismissal for poor instructional performance almost never occurs. Frequency of Teacher Dismissals for Performance (Non-Probationary Teachers)
18 © The New Teacher Project 2009 Agenda About The New Teacher Project Studying Teacher Evaluation The Current Problem The Current Opportunity Preliminary Data About Peer Review Programs How to Build a Better Evaluation System Appendix
19 © The New Teacher Project 2009 Research has shown that teachers have a greater impact on student learning than any other school-based factor. Dallas students who start 3 rd grade at about the same level of math achievement… After 3 EFFECTIVE Teachers After 3 INEFFECTIVE Teachers …may finish 5 th grade math at dramatically different levels depending on the quality of their teachers. Original analysis by the Education Trust. Source: Heather Jordan, Robert Mendro, and Dash Weerasinghe, The Effects of Teachers on Longitudinal Student Achievement,
20 © The New Teacher Project 2009 Teacher Effectiveness (e.g., Value Add, Growth, Eval. Rating) Boost effectiveness of all teachers through effective evaluation and targeted professional development Improve or exit persistently less effective teachers and replace with more effective teachers. Retain and leverage most effective teachers Optimize new teacher supply by hiring from preparation programs whose teachers consistently achieve higher student outcomes. Increase the concentration of effective teachers in high-need schools. 3 Current Performance Potential Performance Therefore, dramatic improvements in student achievement cannot occur without a sustained focus on maximizing teacher effectiveness. 5 Goals for Optimizing Teacher Effectiveness
21 © The New Teacher Project 2009 TNTP Human Capital Continuum Education leaders must use clear evaluations of teacher effectiveness to inform decisions at each step of the human capital continuum. Supportive policy context that removes barriers and publicizes results / outcomes data Optimizing Supply Managing Effectiveness Recruitment Selection Training / Certification Hiring / Placement On- Boarding School-Level Human Cap. Mngmnt. Evaluation / Prof. Dev. Working Conditions Retention / Dismissal Compensation Evaluation system that differentiates teachers based on student academic growth
22 © The New Teacher Project 2009 Most of the districts we studied considered teacher effectiveness only when making decisions about remediation and dismissal.
23 © The New Teacher Project 2009 Improvement: All teachers should report receiving ongoing feedback that helps them improve, and there should be evidence of that improvement in their observations and in student performance. Higher Standards: Teachers should see a high standard of instruction being upheld for themselves and others, and they should not report instances of poor performance going unaddressed. Credibility: Teachers should feel evaluations are accurate and credible. Differentiation: Each school should know which teachers are currently performing at great, good, fair, and poor levels. Recognition: Examples of outstanding instruction should be recognized and made available to other teachers to help them improve. Support: Professional development should be aligned to the evaluation process. Outcomes of an Effective Teacher Evaluation System
24 © The New Teacher Project 2009 Agenda About The New Teacher Project Studying Teacher Evaluation The Current Problem The Current Opportunity Preliminary Data About Peer Review Programs How to Build a Better Evaluation System Appendix
25 © The New Teacher Project 2009 For many years, peer review has been an innovative approach to teacher evaluation, including in Toledo, a Widget Effect study site. Components of peer review: Early career mentoring and evaluation as a component of the tenure granting process. Joint labor-management decision-making about renewal, support for struggling teachers and teacher dismissal, usually through the use of lead or master teachers. Intervention for poor performance where teachers or principals refer tenured teachers for intervention when necessary. Beliefs guiding peer review: Teachers have a vested interest in the quality of the workforce and will hold their peers to even higher standards than administrators would. The best way to ensure quality is to provide intensive support and feedback to beginning teachers and create avenues for intervention with struggling veterans. Teachers and administrators will make the appropriate referrals for intervention when they observe cases of poor performance.
26 © The New Teacher Project 2009 Compared to teachers in other districts we studied, teachers in Toledo are very satisfied with the supports they receive. *Source: Survey of 14,611 teachers in 11 districts, including 773 teachers in Toledo Public Schools. Percent that responded “Strongly agree” or “Agree.” **Source: Survey of 13,664 teachers in 10 districts, including 773 teachers in Toledo Public Schools. Percent that responded “Strongly agree” or “Agree.” Districts studied in The Widget Effect Toledo
27 © The New Teacher Project 2009 But Toledo ranks in the middle among districts whose teachers believe the evaluation system accurately assesses them and helps them improve their instruction. *Source: Survey of 14,611 teachers in 11 districts, including 773 teachers in Toledo Public Schools. Percent that responded “Strongly agree” or “Agree.” **Source: Survey of 14,611 teachers in 11 districts, including 773 teachers in Toledo Public Schools. Percent that responded “Strongly agree” or “Agree.” Toledo Districts studied in The Widget Effect
28 © The New Teacher Project 2009 About half of Toledo teachers say there is at least one tenured teacher in their school whose instruction is poor enough to warrant dismissal. Source: Survey of 15,176 teachers in 12 districts, including 773 teachers in Toledo Public Schools. Percent that responded “Yes.” Districts studied in The Widget Effect Toledo
29 © The New Teacher Project 2009 According to data from the Toledo Federation of Teachers, the peer review program resulted in 26 departures over a four period. Non-Tenured Teacher DismissalsTenured Teacher Dismissals School YearFormalInformalFormalInformal TOTAL tenured teachers exited annually through formal and informal means, on average. The average number of teachers in Toledo during this time was 2,084.
30 © The New Teacher Project 2009 And less than half of Toledo teachers believe the district maintains high instructional standards—among the lowest percentages of the districts we studied. *Source: Survey of 14,611 teachers in 11 districts, including 773 teachers in Toledo Public Schools. Percent that responded “Strongly agree” or “Agree.” **Source: Survey of 15,176 teachers in 12 districts, including 773 teachers in Toledo Public Schools. Percent that responded “Yes.” Toledo Districts studied in The Widget Effect
31 © The New Teacher Project 2009 Continue visits throughout a teacher’s career, whether they are referred or not. Conduct random observations to audit the quality of instruction, and refer teachers to PAR if performance is poor. Actively canvas teachers about the quality of instruction in their team, grade level and school. Do not wait for teachers to come forward and report problems. Apply multiple ratings rather than only satisfactory and unsatisfactory. Nuanced provides richer feedback, allows progress to be tracked, and recognizes excellence. Use PAR for more than determining which teachers are dismissed. The most comprehensive evaluation systems drive decisions about teacher development, assignment, promotion, compensation, etc. We recommend several strategies to ensure peer review is as effective an evaluation system as possible.
32 © The New Teacher Project 2009 Agenda About The New Teacher Project Studying Teacher Evaluation The Current Problem The Current Opportunity Preliminary Data About Peer Review Programs How to Build a Better Evaluation System Appendix
33 © The New Teacher Project 2009 Evaluator Staffing Staffing structures that allow evaluators to dedicate the time necessary to observe and evaluate teachers. Multiple Evaluation Criteria Teacher contributions to student learning. Teacher actions, as compared to clearly articulated criteria that reflect what exemplary performance looks like at a given experience level. District Policies Shift away from centralized PD to tailored school- level development. Requirement that teachers’ evaluations drive renewal, compensation, career path opportunities, and other rewards / consequences. Probationary periods that last long enough to assess effectiveness comprehensively. Reasonable documentation requirements. Efficient processes for recognizing excellence and addressing ineffectiveness. Strong evaluation systems are based on a comprehensive set of strategies. School and District Leader Accountability Accountability for school and district leaders for conducting accurate, useful performance evaluations and improving teacher performance. Processes and checks to ensure accuracy of evaluations (e.g., tenure review board). Evaluator Training and Support Training and ongoing support for administrators and other evaluators to ensure consistency in evaluations. Training on how to use evaluations to inform feedback and individualized developmental interventions.
34 © The New Teacher Project 2009 Trajectory Low High Effectiveness LowHigh Teacher evaluations should assess both effectiveness and trajectory. Place on a rigorous improvement plan or replace Recognize and appreciate, but offer targeted support Renew for another year, but expect more Reward, retain, and provide chances to lead from the classroom Effectiveness : The teacher’s positive impact on student learning during the evaluation period Trajectory : The rate of improvement the teacher has demonstrated during the evaluation period
35 © The New Teacher Project 2009 Expectations should increase with experience, and tenure should be awarded only if a teacher meets those expectations consistently. Effectiveness Year 1Year 2Year 3Year 4Year 5 Minimum expectations for first year teachers to get renewed Minimum expectations for teachers to get tenure Duration of District’s Probationary Period (e.g., 5 years) Expectations The level of effectiveness at which the teacher’s performance could hold steady and still fully meet expectations in the future.
36 © The New Teacher Project 2009 Research suggests that precise language will improve calibration of raters. Less PrecisionMore Precision Criteria that are articulated in such a vague or confusing manner that we are not confident principals are able to minimize subjectivity and make repeatable assessments. Criteria that are oversimplified, too high level, and not actionable, so much so that one can interpret excellence to be very different things (e.g., may allow lower expectations, rather than higher expectations). Criteria that are so poorly worded that it is unclear what is being evaluated. Criteria that evaluate something entirely different from what it can reasonably assumed it was intended to measure. Criteria that bundle multiple, distinct, and elements criteria together and force the principal to make one evaluation. Criteria that are articulated in such a way (e.g., with precise and clear language) that principals can use the criteria to make repeatable evaluations while minimizing false positives. Criteria that are precisely defined for the teacher and the principal. o The criteria help principals know what to look for and minimize interpretation and subjectivity. o The criteria help teachers know what is expected of them, and know what excellence looks like. “One way to increase reliability is to ensure that the evaluation instrument has clearly defined, non-subjective criteria that require minimal interpretation.” (NCCTQ, 2008)
37 © The New Teacher Project 2009 Agenda About The New Teacher Project Studying Teacher Evaluation The Current Problem The Current Opportunity Preliminary Data About Peer Review Programs How to Build a Better Evaluation System Appendix
38 © The New Teacher Project 2009 About The New Teacher Project The New Teacher Project (TNTP) helps school districts and states fulfill the promise of public education by ensuring that all students—especially those from high-need communities—get excellent teachers. A national nonprofit organization founded by teachers, TNTP is driven by the knowledge that although great teachers are the best solution to educational inequality, the nation’s education systems do not sufficiently prioritize the goal of effective teachers for all. In response, TNTP develops customized programs and policy interventions that enable education leaders to find, develop and keep great teachers and achieve reforms that promote effective teaching in every classroom. Since its inception in 1997, TNTP has recruited or trained approximately 33,000 teachers—mainly through its highly selective Teaching Fellows™ programs—benefiting an estimated 4.8 million students. TNTP has also released a series of acclaimed studies of the policies and practices that affect the quality of the nation’s teacher workforce, most recently including The Widget Effect: Our National Failure to Acknowledge and Act on Differences in Teacher Effectiveness (2009). Today TNTP is active in more than 25 cities, including Baltimore, Chicago, Denver, New Orleans, New York, and Oakland, among others. For more information, please visit
39 © The New Teacher Project 2009 For more information:
© The New Teacher Project 2009 Principles of Teacher Evaluation Design Illinois Performance Evaluation Advisory Committee June 11, 2010.
SMHC Task Force Meeting| August © The New Teacher Project 2009 Agenda Introduction Study Overview and Methodology The Widget Effect in Teacher.
© The New Teacher Project 2009 Teacher Evaluations and Support for Effective Instruction Florida Department of Education What’s Working Series Tampa, Florida.
Effective Teachers Core Initiative of the Houston Independent School District Council for the Great City Schools Fall Conference October 21, 2010 Presenters:
LOUISIANA DEPARTMENT OF EDUCATION Compass: Louisiana’s Educator Support & Evaluation System Louisiana Reading Association Meeting April 21, 2012.
Understanding Student Learning Objectives (S.L.O.s)
Teacher Evaluation System LSKD Site Administrator Training August 6, 2014.
1 Phase III: Planning Action Developing Improvement Plans.
Southern Regional Education Board 1 Preparing Students for Success in High School.
Briefing: NYU Education Policy Breakfast on Teacher Quality November 4, 2011 Dennis M. Walcott Chancellor NYC Department of Education.
Fairmont Catholic Grade School Quality Assurance Review Team School Accreditation.
Promoting Regulatory Excellence Self Assessment & Physiotherapy: the Ontario Model Jan Robinson, Registrar & CEO, College of Physiotherapists of Ontario.
1 DPAS II Process and Procedures for Teachers and Specialists Developed by: Delaware Department of Education.
1 Principal Turnover,Teacher Turnover and Quality, and Student Achievement Ed Fuller, PhD University Council for Educational Administration and The University.
1 Quality Indicators for Device Demonstrations April 21, 2009 Lisa Kosh Diana Carl.
1 © The New Teacher Project (TNTP) works to end the injustice of educational inequality by providing excellent teachers to the students who need.
Mississippi Statewide Teacher Appraisal Rubric (M-STAR) Joint Education Committee Meeting December 12,
Pennsylvania Value-Added Assessment System (PVAAS) High Growth, High Achieving Schools: Is It Possible? Fall, 2011 PVAAS Webinar.
NJDOE TALENT DIVISION OVERVIEW prepared for: New Jersey Association of School Administrators April 30,
COMPENSATION AND BENEFITS STUDY March 21, CONTENTS.
Virginia Teacher Performance Evaluation System 0 August 2012.
1 Approaches to Implementing the 2% Cap for Adequate Yearly Progress NCES Summer Data Conference Washington, DC July 2008.
Commission Briefing Low Performing Schools in Urban High Poverty Communities November 11, 2014 House Committee on Education.
1 Welcome to Module 7 Assessment and Evaluation. 2 Getting Started “From their earliest school experience, students draw life- shaping conclusions about.
The Framework for Teaching Charlotte Danielson 4c: Communicating with Families 1 6/12/201 3.
1. 2 The San Jacinto Unified School District presents: Strategic Plan For
Teacher Evaluation and Rewards OECD Mexico Joint Workshop December 1-2, 2009 Susan Sclafani National Center on Education and the Economy.
Teacher Evaluation and Pay for Performance Michigan Education Association Spring 2011.
Principal Evaluation in Massachusetts: Where we are now National Summit on Educator Effectiveness Principal Evaluation Breakout Session #2 Claudia Bach,
Illinois Department of Children and Family Services, Pathways to Strengthening and Supporting Families Program April 15, 2010 Division of Service Support,
Tell Survey May 12, To encourage large response rates, the Kentucky Education Association, Kentucky Association of School Administrators, Kentucky.
Reform Model for Change Board of Education presentation by Superintendent: Dr. Kimberly Tooley.
Title I, Part A Targeted Assistance 101 Field Services Unit Office of School Improvement.
Iowa’s Teacher Quality Program. Intent of the General Assembly To create a student achievement and teacher quality program that acknowledges that outstanding.
HEE Hui For Excellence in Education June 6, 2012
PAULDING COUNTY SCHOOL DISTRICT STAKEHOLDER’S MEETING February 4, 2013.
Growth, Not Gotcha: Evaluating and Supporting Beginning Teachers INTC 8 th Annual Induction and Mentoring Conference February 26, 2013 Liam Goldrick Director.
NYC DOE – Office of Teacher Effectiveness A. Reflecting on Good Teaching 1.
Session 2: Introduction to the Quality Criteria. Session Overview Your facilitator, ___________________. [Add details of facilitators background, including.
2009 STRATEGIC PLANNING PLAYBOOK April 2, 2009 Prepared by M3.
School Leadership that Works: From Research to Results Marzano, Waters, and McNulty 2005.
IMPLEMENTING EABS MODERNIZATION Patrick J. Sweeney School Administration Consultant Educational Approval Board November 15, 2007.
New Haven, A City of Great Schools MOVING FROM COMPLIANCE TO COHERENCE IN EVALUATION AND DEVELOPMENT: THE IMPACT OF THE E3 PROGRAM NEW HAVEN PUBLIC SCHOOLS.
Using outcomes data for program improvement Kathy Hebbeler and Cornelia Taylor Early Childhood Outcome Center, SRI International.
1 Science Subject Leader Training Preparing new and aspiring Subject Leaders for the role. East Riding/North Yorkshire LAs.
1 (c) 2008 The McGraw Hill Companies Redesigning Teacher Salary Structures School Finance: A Policy Perspective, 4e Chapter 12.
11 Designing and Building a Results-Based Monitoring and Evaluation System: A Tool for Public Sector Management.
Leon County School District Oral Exit Report Quality Assurance Review Team District Accreditation.
C.O.R.E Creating Opportunities that Result in Excellence.
Organizational Behavior (MGT-502) Lecture-43. Summary of Lecture-42.
© 2017 SlidePlayer.com Inc. All rights reserved.