RTI International is a trade name of Research Triangle Institute Early Findings from the Implementation and Impact Study of Early College High School Larry Bernstein - RTI Ryoko Yamaguchi and Fatih Unlu - Abt Associates Julie Edmunds - SERVE Elizabeth Glennie - RTI John Willse, UNC - Greensboro and Nina Arshavsky -SERVE SREE Conference 2010 March 4, 2010

 Too many students not graduating from high school  Of those graduating, many not prepared for postsecondary education and world of work  Of every 100 students entering 9 th grade in public HS in North Carolina, only 70 graduate within 5 years  Only 42 of them enroll in college  Only 19 of them complete a two-/four-year degree within 6 years of graduating from HS  NC response: extensive public-private effort to redesign HSs to make them more effective for all students Background/Context

Model  National Early College High School (ECHS) Initiative – Gates and other funders  Goal to increase number of students graduating from HS prepared for college/work  Targeted at traditionally underrepresented populations of students  Study focuses on North Carolina’s effort, supported by North Carolina New Schools Project  NC has over 60 schools across the state  In NC, ECHSs are small schools, located on college campuses, grades 9-12 or 9-13

ECHS Design Principles Improved student achievement Increased graduation rates Increased enrollment in college Personalization Academic and affective supports Supportive relationships Intermediate OutcomesLong Term Outcomes Increased student attendance Improved attitudes toward self and school Increased frequency of higher level courses Increased aspirations toward college Professionalism Ongoing professional development Collaboration among staff Collective responsibility and decision-making College Ready Articulated program of study, grades 9-12 or 13 leading to Associate’s degree College readiness activities Powerful Teaching and Learning High-quality, rigorous, and relevant instruction Student collaboration and discussion Formative and multiple assessments Common standards Increased graduation from college Improved behavior Purposeful Design: Autonomous governance Located on college campus Small size Flexible use of time Integration with college

Study Overview  Partnership between SERVE, NCDPI, NC New Schools Project, Duke University, UNCG, Abt Associates, RTI International  Funded through federal IES grant  Four-year experimental study comparing students who applied to and were randomly accepted into ECHS with students who applied for and were randomly not accepted (attended regular HS)  Participating schools use lottery to select students out of eligible applicant pool; many lotteries stratified to place priority on target population

Study Goals Need to determine if and how ECHS is effective: 1.Examine impact of model on student outcomes including: attitudes, attendance, achievement, course- taking, and school leaving/dropout rates 2.Determine whether impact varies by student characteristics, such as: gender; race/ethnicity; poverty status; first generation college status; and prior achievement 3.Determine whether specific program components are associated with better student outcomes

Overview of Methodology  Experimental design –Schools determine eligible population –Lottery used to allocate spots –Lottery may be stratified if needed; analyses incorporate weights to reflect differential probabilities of selection  Outcome data collected on both treatment and control students  Implementation data collected primarily on enrolled students –Some information on control group students’ experiences also collected

Lottery Sample (Study Begins) # of Sites # of Students ,1171,403 Total: 34 cohorts in 20 sites 3,244 students

Data Collected on both ECHS (Treatment) and Traditional (Control) Extant Data from School Records  Test scores  Course-taking  Attendance  Dropouts  Discipline Original Student Opinion Survey  Attitudes and engagement  Behavior  School experiences: –Rigorous and relevant instruction; assessment –Affective and academic support –Relationships –Expectations

Data Collected only on ECHS Staff and Students  Implementation Surveys—All Staff and Students –Students—same content as Student Opinion Survey –Staff—questions on implementation of the design principles –Completed annually  Site visits –Interviews with staff and students –Observations –Interviews with college faculty –Occurs once

School extant data: th grade students in 8 cohorts in 6 sites Student Opinion Survey: 575 9th Graders in 10 sites Implementation data:  Surveys from 937 students and 95 staff members in 11 schools  Site visits to 9 schools; 11 more planned Samples

Background Characteristics of Current Ninth Grade Sample TreatmentControl Black 22%21% Hispanic 6%5% Retained 1%*5%* First Generation 44%49% 8th Grade Math Pass Rate 85%*78%* 8th Grade Algebra Take Up 28%22% * significant at p ≤.05; will need to account for differences in analyses. Sample: 718 students in 8 cohorts in 6 schools

Ninth Grade Results Impacts on:  College preparatory course-taking and progression  Students’ behavior and attitudes  Students’ high school experiences

Why focus on course-taking?  Big difference between ECHS and Traditional: expectation that everyone goes to college  Students who don’t take Algebra I by end of 9 th grade will find it much harder to complete college-preparatory course of study  Higher level math courses not required for graduation but test scores included in school accountability → traditional schools may steer students away from those courses  Therefore math is particularly sensitive indicator  Look at course-taking patterns in other subjects as well

Course-taking Findings  By end of 9 th grade, significantly more ECHS students taking and successfully progressing through college preparatory courses  ECHS overall reducing performance gaps due to SES characteristics → leveling of playing field

Impact on Algebra I *Significant at p≤.05

Impact on Geometry

Impact on Algebra II *Significant at p≤.05

Impact on College Prep Mathematics Course-taking *Significant at p≤.05

Impact on Civics and Economics *Significant at p≤.05

Impact on English I

Minimizing Gaps in Course Progression GroupMinorityNon-MinorityGap Treatment79.4%81.5%-2.1% Control57.3%70.9%-13.6% Algebra I Progression Rates for Minority and Non-Minority Students

Gaps in Course Progression by Group Min. vs. Non-Min.1 st. Gen. vs. Non-1 st Gen. FRL vs. Non-FRL Alg. I. Eng. IAlg. I Eng. I

Impact on Students’ Behavior, Attitudes and Experiences  ECHS students missed significantly fewer days of school (unexcused)  ECHS students were less likely to be suspended  ECHS students reported significantly higher levels of academic engagement and greater self-efficacy in math  Overall, ECHS students reported significantly more positive experiences than control students

Impact on Student Attendance *Significant at p≤.05

Impact on Student Behavior *Significant at p≤.05

Impact on Student Attitudes  Any overall impacts in favor of ECHS  Impacts vary by school  ECHS students report higher levels of academic engagement than students in control group (effect size =.3)  Positive impact on self-efficacy in math (effect size =.2)  No impact on other outcomes: –Self-efficacy in English –Persistence (e.g. continuing to do work, etc.)

Looking at Implementation  Collecting data on Design Principles: 1.College Ready 2.Powerful Teaching and Learning 3.Personalization 4.Professionalism 5.Purposeful Design  Data from Student Opinion Survey  Data from Implementation Survey and site visits

Implementation Findings  Overall, ECHS students experience more positive school environments than students in control group  Level of implementation overall high but varies by school  No apparent pattern in level of implementation by design principles

Students’ Experiences ScaleEffect Size High Expectations+.70 Relationships+.40 Challenge+.38 Relevant Instruction+.51 Rigorous Instruction+.51 Academic and Social Support+1.07 Source: Student Opinion Survey, administered to treatment and control

Conclusions North Carolina's ECHS model is creating a more positive school environment Students in this environment have fewer absences and fewer suspensions ECHS students more likely to be on-track for college (defined as taking and succeeding in the required courses) Results suggest that ECHS model also most effective for target populations of students traditionally underrepresented in college

For More Information Larry Bernstein