1. Don Boyd, Pam Grossman, Karen Hammerness, Hamp Lankford, Susanna Loeb, Matt Ronfeldt & Jim Wyckoff www.teacherpolicyresearch.org This work is supported by IES Grant R305E6025. The views expressed may not reflect those of the funder. Recruiting Effective Math Teachers, How Do Math Immersion Teachers Compare?: Evidence from New York City

2. New Math Certified Teachers Hired in New York City, by Pathway, 2002-2008

3. Research Questions  How does the preparation of Math Immersion teachers compare to math teachers entering through other pathways?  How do the achievement gains of the students taught by Math Immersion teachers compare to those of students taught by math teachers entering through other pathways?  How does the retention of Math Immersion candidates compare to math teachers entering through other pathways?

4.  Achievement effects of alternate route teachers comparable to traditional preparation programs on average (Decker et al., 2004 (RCT); Boyd et al., 2006; Kane et al., 2007; Harris and Sass, 2008; Constantine et al., 2009 (RCT))  TFA in NC high schools exceeds other paths (Xu et al., 2007)  More limited work on aspects of preparation that may make a difference (Constantine et al., 2009; Boyd et al., 2009 and Harris and Sass, 2007) Prior Research

5. Data Collection  Program analysis – State documents, program documents, accreditation reports, interviews, surveys, course syllabi; – 5 Math Immersion programs,18 institutions, and TFA that prepare most traditional route teachers for NYC schools  Surveys – 603 new NYC middle and high school math teachers (2005) – Questions about their preparation in math– e.g, opportunities to learn math content, math methods, etc.  Administrative data – All NYC teachers 2004-2008; rich measures of teacher qualifications, including certification exams and areas, teacher retention. – Student achievement 2004-2008; value-added scores in math and ELA, grades 6-8 linked to teachers. – Data on schools and students

6. Effect of Preparation Pathways General specification A igcst =  0 +  1 A ig-1cst-1 +X igcst  2 +C gcst  3 +T gcst  4 + P gcst  5 +  s +  igcst Achievement as a function of: prior achievement, student characteristics classroom characteristics teacher characteristics (sometimes) Preparation pathway (e.g., math immersion) Student or school fixed-effects random error

7. Attributes of Students Taught by First Year Grade 8 Math Teachers by Pathway, 2006 Student AttributesCRNYCTFNYCTF- MI TFAother Lagged Math Achievement 0.238-0.125-0.051-0.139-0.061 Proportion Black0.2920.2770.3220.4420.403 Proportion Hispanic0.3580.4960.4930.5270.372 Proportion Free Lunch 0.5470.6640.6350.6190.66 Classsize27.627.826.926.326.1 Lagged Student Absences 12.313.413.114.813.5 Lagged Suspensions 0.0370.0640.0620.0230.042

8. Attributes of Entering Math Certified New York City Teachers by Pathway, 2004-2008 CRNYCTF-MITFA Teacher AttributesHigh School Middle School High School Middle School High School Middle School Female0.6480.7320.4790.5460.4920.551 Black0.0730.1050.1420.2000.0820.141 Hispanic0.0650.0460.0850.0740.0660.043 Age29.728.931.13023.623.5 Last Score255251274271279 CST Math Score262251257251268269 SAT Math600556616589710648 SAT Verbal506483577564627623 Number of Teachers 47815710985426198

9. Effect of Preparation Paths Relative to NYCTF-MI 13579 PathwaysLevel College Recommend0.0160.0050.0170.0040.006 [1.86][0.47][2.60]**[0.40][0.55] NYC Teaching Fellows0.0210.0220.0230.0150.012 [1.87][1.68][2.74]**[1.38][0.85] Teacher for America0.0550.0180.0680.0320.046** [3.71]**[0.86][5.74]**[1.88][2.77] Other-0.011-0.003-0.0040.002-0.02 [1.27][0.28][0.66][0.27][-1.74] NYCTF-MI Below -0.044 [-1.52] NYCTF-MI NA -0.014 [-1.04] Teacher controls  School fixed effects Student fixed effects

10. Distribution of Teacher Value Added by Pathway, with Empirical Bayes Shrinkage, 2004-2008 TeachersTeachers

11. Effect of Pathways and Experience Relative to Math Immersion of Same Experience, Grades 6-8, 2004-08* No Teacher Controls Experience Pathway1234+ College Recommending0.0180.0240.0100.028 [1.60][1.90][0.65][1.53] NYCTF0.0110.0100.0050.065 [0.74][0.58][0.24][2.76]** TFA0.0540.0560.0410.048 [3.13]**[2.64]**[1.09][1.29] Other-0.028-0.032-0.0180.009 [2.22]*[2.61]**[1.29][0.50] * Same variables as model 1 above.

12. Effect of Pathways and Math Immersion Programs 2004-08, Relative to NYCTF-MI Program Z* Pathway and ProgramLevel College Recommend0.0570.033 [3.94]**[1.89] NYC Teaching Fellows0.0620.047 [3.81]**[2.54]* Teacher for America0.0960.031 [4.96]**[1.21] Other0.0300.027 [2.12]*[1.55] Institution A0.0340.018 [1.50][0.71] Institution B0.0510.029 [2.66]**[1.28] Institution C0.0480.035 [1.71][1.16] Institution D0.0550.037 [2.99]**[1.72] Teacher controls School fixed effects * Same variables as earlier model specification.

13. Teacher Retention by Pathway, Math Certified Teachers, 2004-2008 NYCTF-MI CR Experience TransferLeave TransferLeave 1 12.212.49.613.4 2 18.726.512.319.1 3 23.636.416.027.7 426.542.1 18.031.4 NYCTF TFA Experience TransferLeave TransferLeave 1 8.915.75.08.2 2 16.229.69.958.8 3 19.242.312.175.6 424.447.5 13.278.7

14. Simulation of Average Value Added by Pathway and Experience Accounting for Attrition SimulationAverage Value Added YearNYCTF-MICRNYCTFTFA 10.0000.0180.0110.054 20.0450.0680.0530.103 30.0660.0860.0720.086 40.0520.0810.088 Value Added by Pathway and Experience ExperienceNYCTF-MICRNYCTFTFA 1st year0.0000.0180.0110.054 2nd year0.0510.0750.0610.107 3rd year0.0850.0950.0900.126 4th year0.0630.0910.1280.111

15. Conclusions  MI teachers have about the same value-added as College Recommended teachers  Driven largely by selection, TFA performs much better than either College Rec or Math Immersion  Some evidence that both selection and preparation make a difference  Hypothesis: selective post BA program with tailored coursework that includes content and high quality field experience can meaningfully improve student achievement

16. For papers and surveys: www.teacherpolicyresearch.org

17. Pathways to Teaching in NYC, New Teachers, 2002-08

18. Outline  Research questions  Data and methods  Math preparation in Math Immersion and College Recommending programs  Achievement gains by pathway  Retention by pathway  Summing up

19. The Teacher Workforce and Student Outcomes

20. Campus DCampus Z Teacher AttributesHigh School Middle School High School Middle School Female0.4680.5340.5090.538 Black0.1080.1650.1250.127 Hispanic0.0880.0460.0690.093 Age31.731.929.929.7 Last Score272268277276 CST Math Score255249257 SAT Math618586625616 SAT Verbal573542589586 Number of Teachers 154116322119 Attributes of Entering Math Certified NYCTF-MI Teachers by Preparing Campus, 2004-2008

21. Required Credit Hours for Key Courses by Pathway, 2004-2008 College Recommending Math Courses Math Methods Classroom Manage. Learning Graduate programs Mean 4.935.790.863.75 Standard deviation 5.343.291.832.16 Undergrad prog. Mean 11.004.711.754.50 Standard deviation 11.291.382.262.70 Math Immersion Math Courses Math Methods Classroom Manage. Learning Mean 12.608.400.602.40 Standard deviation 5.772.511.34

22. In your preparation to become a teacher, prior to September 2004, how much opportunity did you have to the following:  learn different ways that students solve particular problems  learn theoretical concepts underlying mathematical applications  explore how to apply mathematical materials to real world problems  learn specific techniques for teaching Algebra (Geometry, Number Theory, Probability and Statistics, Calculus)  learn about typical difficulties students have with Algebra (Geometry, Calculus)  study or analyze student math work  study examples o secondary mathematics teaching in the form of videotapes, written cases, etc.  Practice what you learned about teaching math in your field experience  etc. Survey of 1 st year NYC Teachers—Middle and High School Math

23. Teachers' Perceptions of Preparation by Pathways Relative to NYCTF-MI, ( 2005 Survey of 1 st Year Teachers) Pathway Preparation in Specific Strategies General Opps to Learn Teaching Math Subject Matter Prep in Math College Recommending0.3310.3860.038 [2.99]***[3.54]***[0.33] Teaching Fellows0.274-0.350-0.462 [2.50]**[-3.32]***[-4.12]*** Teach For America0.604-0.007-0.561 [2.74]***[-0.03][-2.48]** Other Path0.0040.3710.320 [0.04][3.31]***[2.74]*** N558543541

24. Estimated Value Added Model* Student Measures Class Average MeasuresExperience Lag score0.593Hispanic-0.1612nd year0.050 [269.33]** [6.81]**[8.92]** Lag score sqrd-0.005Black-0.1523rd year0.082 [3.70]** [6.11]**[12.70]** Female0.010Asian0.0994th year0.091 [6.58]** [3.71]**[12.22]** Asian0.126Class size0.0005th year0.100 [35.45]** [0.85][12.64]** Hispanic-0.059English home-0.0266th year0.096 [19.07]** [1.48][11.01]** Black-0.060Free lunch0.014 [18.21]** [1.57]Pathways Change school-0.078Lagged absent-0.007Coll. Recomm.0.016 [16.22]** [13.30]**[1.86] English home-0.060Lag suspended-0.002NYCTF0.021 [31.51]** [0.15][1.87] Free Lunch-0.017Lag ELA score0.194TFA0.055 [10.46]** [24.73]**[3.71]** Lagged absent-0.005Lag Math score0.076Other-0.011 [64.92]** [9.16]** [1.27] Lag suspended-0.024Std Dev ELA score0.043 [12.20]** [4.78]**N651191 * Also includes student and class ELL status, std dev class math score, indicators for experience through 21 years, year and grade effects

25. Challenges of this type of analysis  Conceptualizing relationships  Research designs  Collecting appropriate data  Achievement tests, tested grades, subjects  Strong controls from administrative data  Other data about teachers  Legal/political  Privacy  Concerns about misuse  Technical/modeling  Models that isolate contribution of teacher attributes

26. Effect of Preparation Paths Relative to NYCTF-MI* 1234 PathwaysLevel College Recommend0.0160.0170.0050.004 [1.86][2.60]**[0.47][0.40] NYC Teaching Fellows0.0210.0230.0220.015 [1.87][2.74]**[1.68][1.38] Teacher for America0.0550.0680.0180.032 [3.71]**[5.74]**[0.86][1.88] Other-0.011-0.004-0.0030.002 [1.27][0.66][0.28][0.27] Teacher controls School fixed effects Student fixed effects * Same variables as earlier model specification.