1. Portability of Teacher Effectiveness across School Settings Zeyu Xu, Umut Ozek, Matthew Corritore May 29, 2016 Bill & Melinda Gates Foundation Evaluation of the Intensive Partnership Sites initiative

2. Motivation  Redistributing effective teachers at the center of several education policy initiatives  Teacher is the most important school input related to student learning  The distribution of effective teachers is uneven (recruiting, who moves, and to where)  Key assumption: Teachers effectiveness is portable  Students face different challenges in learning  School culture, environment and working conditions may affect teacher learning, practices, efforts, burnout, etc.  Literature  Jackson (2010), Jackson & Bruegmann (2009), Goldhaber & Hansen (2010)  Sanders, Wright & Langevin (2009) › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

3. Research Questions  Do teachers retain their effectiveness across schools  On average  Across schools with similar settings  Across schools with different settings (by the direction of the change)  Teacher effectiveness measured by  Value-added  Settings defined by  School performance levels  School poverty levels  Conditional on teachers switching schools › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

4. Preview of Findings  Among teachers who changed schools, on average their VA was unchanged or slightly improved  The same conclusion holds regardless of the similarity/difference between the sending and receiving schools or the direction of the move  High-performing teachers’ VA dropped and low-performing teachers’ VA gained in post-move years  This pattern is mostly driven by regression to the within-teacher mean and has little to do with school moves  Despite this pattern, high VA teachers still performed at a higher level than low VA teachers in post-move years › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

5. Organization  Data and samples  Methodology  Findings  Summary and discussion › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

6. Data  North Carolina 1998-99 through 2008-09  Elementary level (4 th and 5 th grade math and reading teachers, self- contained classrooms)  Secondary level (algebra I and English I teachers, “Algebra I”, “Algebra I-B”, “Integrated Math II”, “English I” classrooms)  Florida 2002-03 through 2008-09  Elementary level (4 th and 5 th grade math and reading teachers, “core courses” in a given subject)  Secondary level (9 th and 10 th grade math and reading teachers, “core courses” in a given subject) › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

7. Sample restrictions  Remove charter schools  Remove students and teachers who changed schools during a school year (about 2-4% of obs)  Remove students with missing values on covariates  Keep classrooms with 10~40 students  Remove classrooms with >50% special education students › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

8. Sample sizes North CarolinaFlorida ElementarySecondaryElementarySecondary Math21,1194,99929,9899,101 Reading21,1193,77529,3549,681 › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 Number of Unique Teachers in the Analytic Samples

9. Two-Stage Analysis  Estimate teacher-year value-added  Difference-in-differences analysis › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

10. Estimate Teacher VA  Test scores standardized by year, grade and subject (mean=0, sd=1)  (X) Covariates include:  1) grade repetition, 2) FRPL, 3) sex, 4) race/ethnicity, 5) gifted, 6) special education, 7) student school mobility and 8) grade level.  Bias (no school FE)  Noise (EB adjustment)  Alternative model specifications (achievement levels model) › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

11. DiD  Three groups: non-movers, movers to a similar school setting, movers to a different school setting  FGLS, se clustered at the teacher level  (Y) Year and (T) teacher FEs  (X) Teacher experience (0-2, 3-5, 6-12, 13 or more years of exp)  (S) School quality (average peer VA)  (C) Classroom characteristics (FRL %, mean pretest score, sd of pretest score)  (Post) Post-move years indicator  (DP, DN) Indicators for school setting differences › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

12. Define School Settings  School performance  NC: % students performing at or above grade level  FL: School performance scores based on both levels and growth  Standardized by year and aggregated across all years  School poverty  % FRPL  Aggregated across all years in which a teacher taught in that school  Change in school setting measures  ∆ = Receiving school – Sending school  Similar setting = within half a SD around the mean of the ∆ distribution  DP = 1 if ∆ > 0.25 (performance) or ∆ > 0.15 (poverty)  DN = 1 if ∆ < -0.25 (performance) or ∆ < -0.15 (poverty) › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

13. Alternative DiD Specs  Last pre-move year and first post-move year  Between- vs. within-district moves  Replace the post-move indicator with individual year dummies (I t-1, I t-2, I t-3 …; I t+1, I t+2, I t+3 ) › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

14. Distribution of Movers › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 By school performance setting change

15. Distribution of Movers › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 By school poverty setting change

16. Mover Characteristics › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 NC elementary school teachers, by mobility status

17. Pre-Post Change in VA (elem) › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 North CarolinaFlorida MathReadingMathReading All0.0040.005-0.0010.002 By school perf. Higher to lower0.0190.0110.002 Similar0.004 0.007-0.001 Lower to higher-0.0020.003-0.0050.004 By school poverty Higher to lower-0.0050.002-0.0040.002 Similar0.0050.0040.0000.002 Lower to higher0.0200.0170.009

18. Pre-Post Change in VA (sec) › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 North CarolinaFlorida MathReadingMathReading All0.0560.003 0.005 By school perf. Higher to lower0.067-0.0110.0030.013 Similar0.0850.0140.0060.008 Lower to higher0.0300.0050.0020.000 By school poverty Higher to lower0.1110.002 -0.006 Similar0.0570.0100.0040.006 Lower to higher-0.010-0.020-0.0030.019

19. By Pre-Move VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 Actual year of move “Pseudo” move

20. By Pre-Move VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 Elementary math teachersElementary math teachers (pseudo move)

21. By Pre-Move VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 Elementary reading teachersElementary reading teachers (pseudo move)

22. By Pre-Move VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 Secondary math teachersSecondary math teachers (pseudo move)

23. By Pre-Move VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 Secondary reading teachersSecondary reading teachers (pseudo move)

24. Adjacent Year Correlations › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 Correlation North CarolinaFlorida MathReadingMathReading Y t-2, Y t-1 0.4830.2980.3800.187 (0.426, 0.535)(0.232, 0.362)(0.314, 0.443)(0.111, 0.260) Y t-1, Y t+1 0.3410.2700.3020.138 (0.256, 0.420)(0.182, 0.354)(0.231, 0.369)(0.061, 0.213) Y t-+1 Y t-+2 0.4630.2690.4270.191 (0.381, 0.537)(0.175, 0.358)(0.363, 0.487)(0.115, 0.264)

25. Pre-Post Comparisons of VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 North Carolina

26. Pre-Post Comparisons of VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 Florida

27. Summary  Among teachers who changed schools, on average their VA was unchanged or slightly improved  The same conclusion holds regardless of the similarity/difference between the sending and receiving schools or the direction of the move  High-performing teachers’ VA dropped and low-performing teachers’ VA gained in post-move years  This pattern is mostly driven by regression to the within-teacher mean and has little to do with school moves  Despite this pattern, high VA teachers still performed at a higher level than low VA teachers in post-move years › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

28. Discussion  Teacher effectiveness does not appear to be hurt by moving to schools with different settings.  Multiple years of VA estimates can be used with other teacher evaluation data to identify effective teachers, capturing persistent teacher performance better and reducing post-move year shrinkage.  All results take teacher school changes as given. › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016