1. Portability of Teacher Effectiveness across School Settings
Bill & Melinda Gates Foundation
Evaluation of the Intensive Partnership Sites initiative
Portability of Teacher Effectiveness across School Settings
Zeyu Xu, Umut Ozek, Matthew Corritore
November 17, 2018

2. Motivation
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
Redistributing effective teachers at the center of several education policy initiatives
Teacher is the most important school input related to student learning
The distribution of effective teachers is uneven (recruiting, who moves, and to where)
Key assumption: Teachers effectiveness is portable
Students face different challenges in learning
School culture, environment and working conditions may affect teacher learning, practices, efforts, burnout, etc.
Literature
Jackson (2010), Jackson & Bruegmann (2009), Goldhaber & Hansen (2010)
Sanders, Wright & Langevin (2009)
November 17, 2018

3. Research Questions
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
Do teachers retain their effectiveness across schools
On average
Across schools with similar settings
Across schools with different settings (by the direction of the change)
Teacher effectiveness measured by
Value-added
Settings defined by
School performance levels
School poverty levels
Conditional on teachers switching schools
November 17, 2018

4. Preview of Findings
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
Among teachers who changed schools, on average their VA was unchanged or slightly improved
The same conclusion holds regardless of the similarity/difference between the sending and receiving schools or the direction of the move
High-performing teachers’ VA dropped and low-performing teachers’ VA gained in post-move years
This pattern is mostly driven by regression to the within-teacher mean and has little to do with school moves
Despite this pattern, high VA teachers still performed at a higher level than low VA teachers in post-move years
November 17, 2018

5. Organization Data and samples Methodology Findings
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
Data and samples
Methodology
Findings
Summary and discussion
November 17, 2018

6. Data North Carolina 1998-99 through 2008-09
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
North Carolina through
Elementary level (4th and 5th grade math and reading teachers, self- contained classrooms)
Secondary level (algebra I and English I teachers, “Algebra I”, “Algebra I-B”, “Integrated Math II”, “English I” classrooms)
Florida through
Elementary level (4th and 5th grade math and reading teachers, “core courses” in a given subject)
Secondary level (9th and 10th grade math and reading teachers, “core courses” in a given subject)
November 17, 2018

7. Sample restrictions Remove charter schools
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
Remove charter schools
Remove students and teachers who changed schools during a school year (about 2-4% of obs)
Remove students with missing values on covariates
Keep classrooms with 10~40 students
Remove classrooms with >50% special education students
November 17, 2018

8. Number of Unique Teachers in the Analytic Samples
Sample sizes
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
Number of Unique Teachers in the Analytic Samples
North Carolina
Florida
Elementary
Secondary
Math
21,119
4,999
29,989
9,101
Reading
3,775
29,354
9,681
November 17, 2018

9. Two-Stage Analysis Estimate teacher-year value-added
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
Estimate teacher-year value-added
Difference-in-differences analysis
November 17, 2018

10. Estimate Teacher VA
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
Test scores standardized by year, grade and subject (mean=0, sd=1)
(X) Covariates include:
1) grade repetition, 2) FRPL, 3) sex, 4) race/ethnicity, 5) gifted, 6) special education, 7) student school mobility and 8) grade level.
Bias (no school FE)
Noise (EB adjustment)
Alternative model specifications (achievement levels model)
November 17, 2018

11. DiD
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
Three groups: non-movers, movers to a similar school setting, movers to a different school setting
FGLS, se clustered at the teacher level
(Y) Year and (T) teacher FEs
(X) Teacher experience (0-2, 3-5, 6-12, 13 or more years of exp)
(S) School quality (average peer VA)
(C) Classroom characteristics (FRL %, mean pretest score, sd of pretest score)
(Post) Post-move years indicator
(DP, DN) Indicators for school setting differences
November 17, 2018

12. Define School Settings
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
School performance
NC: % students performing at or above grade level
FL: School performance scores based on both levels and growth
Standardized by year and aggregated across all years
School poverty
% FRPL
Aggregated across all years in which a teacher taught in that school
Change in school setting measures
∆ = Receiving school – Sending school
Similar setting = within half a SD around the mean of the ∆ distribution
DP = 1 if ∆ > 0.25 (performance) or ∆ > 0.15 (poverty)
DN = 1 if ∆ < (performance) or ∆ < (poverty)
November 17, 2018

13. Alternative DiD Specs Last pre-move year and first post-move year
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
Last pre-move year and first post-move year
Between- vs. within-district moves
Replace the post-move indicator with individual year dummies (It-1, It-2, It-3…; It+1, It+2, It+3)
November 17, 2018

14. Distribution of Movers
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
By school performance setting change
November 17, 2018

15. Distribution of Movers
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
By school poverty setting change
November 17, 2018

16. Mover Characteristics
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
NC elementary school teachers, by mobility status
November 17, 2018

17. Pre-Post Change in VA (elem)
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
North Carolina
Florida
Math
Reading
All
0.004
0.005
-0.001
0.002
By school perf.
Higher to lower
0.019
0.011
Similar
0.007
Lower to higher
-0.002
0.003
-0.005
By school poverty
-0.004
0.000
0.020
0.017
0.009
November 17, 2018

18. Pre-Post Change in VA (sec)
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
North Carolina
Florida
Math
Reading
All
0.056
0.003
0.005
By school perf.
Higher to lower
0.067
-0.011
0.013
Similar
0.085
0.014
0.006
0.008
Lower to higher
0.030
0.002
0.000
By school poverty
0.111
-0.006
0.057
0.010
0.004
-0.010
-0.020
-0.003
0.019
November 17, 2018

19. By Pre-Move VA Actual year of move “Pseudo” move November 17, 2018
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
Actual year of move
“Pseudo” move
November 17, 2018

20. By Pre-Move VA Elementary math teachers
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
Elementary math teachers
Elementary math teachers (pseudo move)
November 17, 2018

21. By Pre-Move VA Elementary reading teachers
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
Elementary reading teachers
Elementary reading teachers (pseudo move)
November 17, 2018

22. By Pre-Move VA Secondary math teachers
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
Secondary math teachers
Secondary math teachers (pseudo move)
November 17, 2018

23. By Pre-Move VA Secondary reading teachers
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
Secondary reading teachers
Secondary reading teachers (pseudo move)
November 17, 2018

24. Adjacent Year Correlations
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
Correlation
North Carolina
Florida
Math
Reading
Yt-2, Yt-1
0.483
0.298
0.380
0.187
(0.426, 0.535)
(0.232, 0.362)
(0.314, 0.443)
(0.111, 0.260)
Yt-1, Yt+1
0.341
0.270
0.302
0.138
(0.256, 0.420)
(0.182, 0.354)
(0.231, 0.369)
(0.061, 0.213)
Yt-+1 Yt-+2
0.463
0.269
0.427
0.191
(0.381, 0.537)
(0.175, 0.358)
(0.363, 0.487)
(0.115, 0.264)
November 17, 2018

25. Pre-Post Comparisons of VA
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
North Carolina
November 17, 2018

26. Pre-Post Comparisons of VA
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
Florida
November 17, 2018

27. Summary
› Introduction
› Data and Samples
› Methodology
› Findings
› Summary and Discussion
Among teachers who changed schools, on average their VA was unchanged or slightly improved
The same conclusion holds regardless of the similarity/difference between the sending and receiving schools or the direction of the move
High-performing teachers’ VA dropped and low-performing teachers’ VA gained in post-move years
This pattern is mostly driven by regression to the within-teacher mean and has little to do with school moves
Despite this pattern, high VA teachers still performed at a higher level than low VA teachers in post-move years
November 17, 2018