1 Comments on: “New Research on Training, Growing and Evaluating Teachers” 6 th Annual CALDER Conference February 21, 2013

2 How Can We Improve Teacher Quality?  Improve the pool of applicants  Alternative routes into teaching  Improve traditional teacher preparation programs  Increase starting salaries  Raise entry requirements  Do a better job of selecting the best candidates  Observable characteristics generally not related to future performance  Improve the human capital of current teachers  Studies of various forms of PD/mentoring typically find no impact on student achievement  Get existing teachers to work harder/better  Nashville and similar merit-pay experiments find little/no impact on teacher performance  Get rid of low-performing teachers and hold on to high- performing teachers  Simulations suggest gains could be large; don’t not yet know how this will pan out in practice

3 Teacher Preparation Programs and Teacher Quality: Are There Real Differences Across Programs?  Short Answer: No (at least in Missouri)  Few programs stand out in other states either  Florida, Louisiana, New York, Texas  Why Aren’t There Differences?  Uniform negative selection  Education majors at more selective universities aren’t much better than those at less selective universities  Maybe teacher prep. programs don’t differ much across institutions  Or maybe they don’t matter at all

4 Do First Impressions Matter? Improvements in Early Career Teacher Effectiveness  Short Answer: Yes  Value of teacher retention/”de-selection” based on value- added (VA) depends on how well early-career VA predicts long-run performance  Although there is substantial variability in the payoff to early career experience, VA in first two years does a reasonably good job at predicting later VA, particularly if you want to identify high and low performers  Still only about 28% of variation in future VA is “explained” by VA in first two years  If dismiss teachers in lowest 10% of VA in first two years, would eliminate 30% of teachers who are later in bottom 10% of VA and none of those who are later in the top 10% based on VA

5 Selecting Growth Measures for School and Teacher Evaluations: Should Proportionality Matter?  Consider three models  Student Growth Percentiles  Account for differences in potential achievement solely with prior achievement  One-Step Value Added  Use both prior test scores and student demographics to account for differences in potential achievement  Simultaneously estimate impacts of student characteristics and schools/teachers by looking at within-school/within-teacher variation in student achievement (i.e. fixed effects)  Two-Step Value Added  First step estimates achievement as a function of a student’s prior scores and characteristics, plus observable school/teacher characteristics  Second step takes the difference between the actual score and the predicted score for each student in step one and compares the averages across schools/teachers

6 Selecting Growth Measures for School and Teacher Evaluations: Should Proportionality Matter?  Student Growth Percentiles  Advantages  Doesn’t explicitly set lower expectations for disadvantaged students  Disadvantages  If disadvantaged students score lower (conditional on prior test scores) than non-disadvantaged students, then penalizing teachers/schools who serve them

7 Selecting Growth Measures for School and Teacher Evaluations: Should Proportionality Matter?  One-Step Value Added  Advantages  Takes into account that some types of students will likely score lower than others, even if they have the same initial score  Disadvantages  Assumes effects of student characteristics within-school (or within-teacher) are the same as the effects across schools (or teachers)  Effects of small differences in student characteristics within schools/teachers extrapolated to effects of large differences in characteristics across schools/teachers  Likely much bigger deal for ranking schools than for ranking teachers  Measurement error a bigger problem since less true variation within schools/teachers

8 Selecting Growth Measures for School and Teacher Evaluations: Should Proportionality Matter?  Two-Step Value Added  Advantages  Does not penalize schools for factors outside their control  Schools serving disadvantaged students may get worse pool of teacher candidates  Disadvantages  If school/teacher quality is correlated with student characteristics, will underestimate differences in quality  Falsely attribute differences in school/teacher quality to student characteristics  If disadvantaged students tend to be assigned low-quality teachers, then end up penalizing high-quality teachers

9 Selecting Growth Measures for School and Teacher Evaluations: Should Proportionality Matter?  What’s the right model?  Ehler, et al. argue for the two-step VA model  Choice not so clear  SGPs appear problematic because they do not account for student characteristics (conditional on test scores) but with enough prior scores, may not be so bad  One-step VAM may over-penalize schools/teachers serving disadvantaged students  Two-step VAM may under-reward schools/teachers serving advantaged students

10 Implications  Improving the pool of potential teachers important, but tweaking traditional teacher preparation programs not likely to get us there  Need to find was to get higher quality potential teachers by lowering entry barriers, increasing incentives or (maybe) radically changing traditional teacher preparation programs  Although far from a perfect instrument, incorporating early- career value added into retention/dismissal decisions will help  While there is no single “right” value added model, states ought to be paying more attention about the consequences of selecting a particular model and consider how they intend to use the results when decided which model they choose