1. Preliminary Highlights from the Noyce National Program Evaluation May 30, 2013 Ellen Bobronnikov Cris Price

2. Abt Associates | pg 2 Study Overview  Implementation study examines the Noyce Program across awards from PI, Faculty, Recipients, and K-12 Administrators’ perspectives –Data sources: Web surveys and interviews  Impact study assesses program’s impact on teacher certification and employment in high-need districts –Data sources: Teacher certification and employment data from 5 states and annual “monitoring” data entered by PIs

3. Abt Associates | pg 3 Selected Implementation Study Research Questions  What activities do teacher preparation programs use to prepare and support Noyce recipients?  What are STEM faculty responsibilities for preparing K–12 mathematics and science teachers?  What activities do Noyce recipients engage in?

4. Abt Associates | pg 4 Noyce Has Funded Recipients Across the Country

5. Abt Associates | pg 5 Number of Recipients First Supported Each School Year

6. Abt Associates | pg 6 Results: Activities to Prepare and Support Noyce Recipients  Noyce recipients in teacher preparation, including TFs, enrolled in same range of courses available/required of all students in teacher preparation –Additional activities were available to Noyce recipients  Supports for Noyce recipients who were teaching included mentoring, PD, courses, conference support  Supports for MTFs included leadership training and educational resources  Supports for interns included hands-on experience with K–12 students, courses to introduce them to teaching, and internships in various settings

7. Abt Associates | pg 7 Results: STEM Faculty Involvement with Noyce  After receiving the Noyce award… –Over half of faculty/PI respondents reported increased STEM faculty member engagement in training STEM K–12 teachers –About a quarter of STEM faculty respondents reported changes in their teaching due to the Noyce Program (e.g., focus on active learning, adapting course content to needs of teachers)

8. Abt Associates | pg 8 Results: Recipient Experiences While in Teacher Prep  Most recipients decided to enter K–12 teaching during or after college  Just under half decided to teach in high-need districts at the time of application to Noyce  Almost all completed student teaching in a STEM subject area, and 80% taught in a high need district  Majority of recipients reported that they felt adequately prepared for teaching responsibilities

9. Abt Associates | pg 9 Results: Recipient Experiences While Teaching  Induction supports were primarily received in the first year of teaching, other than support for conferences, which was received throughout  Common leadership roles held by Noyce teachers included mentoring, committee service, departmental leadership, advising student/school organizations  Almost all Noyce teachers reported that they plan to complete their Noyce teaching obligation and continue teaching science/ math in a high-need school district

10. Abt Associates | pg 10 Results: TF Experiences  TFs must be in a Master’s Program and are required to teach in a high-need district for 4 years while receiving a salary supplement  Four-fifths of TFs reported that they had decided to teach some time during or after college; one-third were career changers  Almost all TFs indicated that their teacher prep program used a cohort model and included K-12 teachers as science/math mentors; half of TFs participated in a PLC during their first year teaching

11. Abt Associates | pg 11 Results: MTF Experiences  MTFs are required to teach in a high-need district for 5 years while receiving a salary supplement  MTFs reported involvement in leadership activities prior to Noyce, but generally reported involvement in more leadership activities post-Noyce  The majority of MTFs planned to retain both classroom AND leadership responsibilities; fewer indicated plans to assume primarily leadership roles or to shift to higher education

12. Abt Associates | pg 12 Results: Intern Experiences  One-third of respondents who had applied for the Noyce internship had not considered teaching prior to learning about Noyce  Common internship settings included math/science camps, research labs, schools, or museums  About half of intern respondents indicated their interest in working as a K–12 teacher increased after participating in the Noyce internship Now, a transition to impact study …

13. Abt Associates | pg 13 Impact Study Research Questions  Does an IHE’s receipt of a Noyce grant affect its production of graduates who are certified by their state to teach STEM content?  Does an IHE’s receipt of a Noyce grant affect its production of certified STEM teachers who take teaching jobs in high-need schools?  What is the impact of Noyce on student achievement?

14. Abt Associates | pg 14 Years to Certification and Teaching  Among all recipients: –Nearly two-thirds had received their teaching certificate  Among those who had enough time to complete program and earn certifications (2+ years from first receipt of support): –83% have been certified to teach  Among recipients who have had at least 2 years to find a teaching position after certification: –90% had taught in high-need districts, in fulfillment of their service requirement

15. Abt Associates | pg 15 Program is being implemented as intended…  Most recipients received STEM certifications and entered teaching in high-need schools (according to monitoring data)  However, this alone does not answer the question: “Would recipients have earned STEM certifications and taught in a high-need schools in the absence of the program?”

16. Abt Associates | pg 16 Preliminary Findings from Teacher Impact Study  Two of the 5 study states had significant positive impact estimates on STEM certification and employment in high-need schools –Impact represents an additional 4-5 teachers per IHE per impact year, about what we would expect from monitoring data  Two study states had impact estimates that were not significantly different than zero  One study state had a large significant negative impact estimate

17. Abt Associates | pg 17 Teacher Impact Study Limitations  All study states except one had small numbers of Noyce IHEs that graduated recipients early enough to be included in state datasets  Year-to-year variation in numbers of recipients who were certified and/or employed within IHEs was often large relative to the expected size of the impact per IHE per year  Impact estimates are NOT from a randomized study –The quasi-experimental comparative short interrupted time series approach that was used to estimate impacts depends on models and assumptions

18. Abt Associates | pg 18 Next Steps  Examine state contexts to better understand the mixed findings in the teacher impact study.  Collect certification and employment data from another large state to include in teacher impact study  Collect an additional year of data from some study states  Conduct student impact analyses in 3 districts  Prepare dissemination report