1. Overview of the Early College High School Initiative Evaluation Susan Cole Mengli Song Andrea Berger American Institutes for Research Presentation at the SREE 2010 Conference March 4, 2010

2. About the ECHSI  Started in 2002 by the Bill & Melinda Gates Foundation o Improve postsecondary access and success o Provide opportunity to earn up to 2 years of college credit  In fall 2009, over 200 Early College Schools (ECSs) open across the nation

3. About the ECHSI  Five Core Principles define an Early College School (ECS) o Target population o Partnership with a college or university o Integrated academic plan o Student supports o Advocate for supportive policies

4. Evaluation Research Questions 1.What are the structural, organizational, and instructional characteristics of ECSs? 2.What are the intermediate and long-term outcomes for students attending ECSs, especially for students traditionally underserved by the postsecondary system?

5. Data Sources and Analytic Methods  Qualitative data o Site visits (6 to 20 ECSs annually)  Quantitative data o School survey (entire ECS population annually) o Student survey (35 schools and 2,102 students in 2007-08)  Analytic Methods o School survey- descriptive statistics and regression o Student survey- hierarchical linear modeling

6. RQ1: Characteristics of ECSs  157 ECSs in 2007-08 across 21 states and DC  65% have a 2-year public college partner  53% are located on a college campus  76% have admissions criteria Fewer than 100 students per grade on average Source: 2007-08 ECHSI school survey

7. RQ1: Who do ECSs serve?  Minority- 67%  Free or Reduced-Price Lunch Eligible- 59%  Limited English Proficiency- 10%  1 st Generation College Going- 46% Sources: 2007-08 ECHSI school survey; 2007-08 ECHSI student survey

8. RQ1: What are characteristics of college classes in ECSs?  91% of ECSs have at least some students in college classes  61% of students have taken at least one college class o Half of these college classes are in core academic areas o 66% of these college classes are taken on a college campus Source: 2007-08 ECHSI school survey; 2007-08 ECHSI student survey

9. RQ2: How are ECS students doing?  On average, ECSs had average daily attendance (ADA) rates over 94%.  74% of ECS students were proficient in ELA; 67% were proficient in math. o ECSs outperformed districts in both ELA and math proficiency rates by 7%. Sources: 2007-08 ECHSI school survey; publicly available school data

10. RQ2: How are ECS students doing?  Grade-to-grade progression rates o 85% on average for 9th-to-10th-grade o 87% for the 12th-to-graduation or grade 13  Cumulative Promotion Index (CPI) o For the 12 ECSs with data, the average CPI was 66% o Exceeded districts’ CPI by an average of 14% Sources: 2007-08 ECHSI school survey; publicly available school data

11. RQ2: How are ECS students doing?  Graduates earned about a semester to a year of college credit while enrolled in the ECS. o ECS survey: about 8 college classes o ECS transcripts: about 10 college classes Sources: 2007-08 ECHSI school survey; 2004-05 to 2007-08 student transcript data

12. RQ2: How are ECS students doing?  Most ECS students enrolled in college after graduation.  The ECS average for college enrollment is equivalent to or exceeds national averages. College Enrollment: ECS & National Average Sources: 2007–08 ECHSI school survey; NCES, 2007 ECS (2006-07) National (2003-04) Postsecondary Enrollment 88%72% 4-year IHE45%44% 2-year IHE43%28%

13. RQ2: How are ECS students doing?  Minor differences between subgroups on various outcomes  1 st generation college-going students most consistent gap o Lower high school and college GPAs o Lower educational aspirations o Lower satisfaction with the ECS Sources: 2007-08 ECHSI student survey; 2007-08 ECHSI school survey; publicly available school data

14. RQ2 How are ECS students doing?  Students at ECSs located on a college campus had higher outcomes than ECSs at other locations on: o ADA (95% and 93%) o 9 th - to 10 th -grade progression rates (89% and 81%) o Achievement proficiency rates (relative to their district) ELA- 14% above and 1% below Math- 16% above and 1% below Sources: 2007-08 ECHSI student survey; 2007-08 ECHSI school survey; publicly available school data

15. Summary of findings What do we know about ECS students?  Students are largely from populations underrepresented in postsecondary institutions.  ECS students are outperforming districts on state assessments.  Students are accumulating college credit.  On many outcomes, students in ECSs located on college campuses are doing better than student attending ECSs not located on college campuses. Sources: 2007-08 ECHSI student survey; 2007-08 ECHSI school survey;

16. ECHSI Impact Study: Research Questions 1.Do ECS students have better outcomes than they would have had at other high schools? 2.Do the effects of ECSs on student outcomes differ for different types of schools? 3.Do the effects of ECSs differ for students with different background characteristics?

17. ECHSI Impact Study: Design and Sample  Overall Study Design: 3-year multisite RCT  Sample: – Sites: Up to 21 ECSs that use lottery-based admission in at least one year between 2003-04 and 2007-08 – Students (up to 5 cohorts per school):  Treatment: offered space in ECS through lottery  Comparison: not offered space in ECS through lottery  Extant data sample: about 4,600 treatment and 6,700 comparison students across 21 sites  Student survey sample: about 1,800 students

18. ECHSI Impact Study: Outcome Measures  High School Outcomes: – High school persistence and graduation – ACT/SAT test taking and performance – College credit accrual while in high school  College Outcomes: – Highest educational attainment – Degrees and certificates earned

19. ECHSI Impact Study: Data Collection  Extant Data Collection: – Data: lottery records, student and school characteristics and student outcomes – Sources: ECSs, districts, and/or subcontractors – March 2010 through November 2011  Student Survey: – Designed to capture additional student outcome information for all sites – The only source for student outcomes for some sites – June to November 2011

20. ECHSI Impact Study: Analytic Methods for RQ1 RQ1 (Overall Impact):  Intent-to-treat analyses  Multilevel models with sites as random effects (students nested within sites) – HGLM for binary outcomes – Multilevel survival analyses – HGLM for multinomial outcomes

21. ECHSI Impact Study: Analytic Methods for RQs2 &3 RQ 2 (Differential Impact on different types of schools):  Assessed by incorporating measures of ECS characteristics as site-level predictors into the main impact model RQ 3 (Differential Impact on different types of students):  Assessed by incorporating ECS-by-student- characteristic interactions as student-level predictors into the main impact model