1. Human Capital Policies in Education: Further Research on Teachers and Principals 5 rd Annual CALDER Conference January 27 th, 2012

2. Estimating the Effect of Leaders on Public Sector Productivity: The Case of School Principals Gregory Branch, Eric Hanushek, and Steven Rivkin, January 2012

3. Questions Is there substantial variation in principal effectiveness? Does the variation in principal effectiveness differ by the share of low income students in a school? Do more effective principals make better personnel decisions? Are “effective” principals more likely to leave high poverty schools?

4. UTD Texas Schools Project Stacked panels of students and staff Annual student testing Student demographic characteristics Information on staff  Follow principals, teachers, and students in Texas public schools  Very large samples: 7,420 unique principals and 28,147 principal-year observations in 1995- 2001

5. Principals Change Frequently Quartile Principal in 1 st year Principal with 6+ years Poverty lowest17.836.1 highest19.531.6 Math achievement worst22.726.3 best16.438.8

6. Estimation of Variation in Principal Quality Non-random selection of principals and students Control for observed student characteristics and prior achievement Make principals comparable in terms of tenure

7. No Simple Solutions Alternative approaches to estimation 1. Fixed effects for principals 2. Fixed effects for principals and schools 3. Direct estimation of quality variation 4. Validation with teacher turnover analysis

8. Alternative Value-Added Estimates Principal Spell Fixed Effects Regress math score on lagged math score, student demographic variables, principal-by-spell fixed effects

9. Alternative Value-Added Estimates Principal Spell Fixed Effects Regress math score on lagged math score, student demographic variables, principal-by-spell fixed effects Principal Spell and School Fixed Effects

10. Fixed Effects Estimates (s.d.) (without school fixed effects) Poverty quartile First three years tenure Lowest0.16 2 nd 0.18 3 rd 0.21 Greatest0.26 All0.21

11. Test Measurement Issues Random measurement error  Use Bayesian shrinkage estimator Basic Skills Tests  Reweight to allow for initial achievement

12. Measurement Effects (s.d.) First three years tenure Unadjusted0.21 Shrunk0.20 Re-weighted0.27 Shrunk/re-weighted0.24

13. Alternative Fixed Effects Estimates (s.d.) Poverty quartile Up to six years tenure Lowest0.18 2 nd 0.19 3 rd 0.23 Greatest0.28 All0.22

14. Alternative Fixed Effects Estimates (s.d.) Poverty quartile Up to six years tenure With school fixed effects Lowest0.180.08 2 nd 0.190.10 3 rd 0.230.12 Greatest0.280.14 All0.220.11

15. Why is variance higher in high poverty schools? Larger variation in underlying principal skills in high poverty schools Or Principal quality differences translate into larger differences in test scores in high poverty schools

16. Direct Estimates of Variance If principal changes and if principal effects outcomes, pattern of student growth should change If other school factors are uncorrelated with principal change (partially testable), can obtain lower bound estimate of principal effectiveness.

17. Direct Lower Bound Estimates Adjacent yearInterrupted year With student controls* Different principal 0.0510.0500.054 *Student ethnicity, gender, SES, ELL, special education, mobility measures

18. Range of Alternative Estimates Principal Fixed Effect Principal and School Fixed Effect Direct Estimates TotalWithin school Standard Deviation 0.210.110.050

19. Added Analysis – Principal Quality 1.Better principals => better teacher transitions 2.High mobility of both best and worst in most disadvantaged schools 3.Substantial number of worst principals become principal elsewhere.

20. Summary Purposeful sorting complicates estimates of principal quality and quality of leavers Substantial variation in estimates of principal quality (fixed effects and direct) Higher variance in high poverty schools Not due to test measurement complications Effects are large

21. www.utdallas.edu/research/tsp-erc