Does Remediation Help All Students Does Remediation Help All Students? How the Effects of Postsecondary Developmental Courses Vary by Background and Ability Bridget Terry Long Angela Boatman Harvard Graduate School of Education IES Conference June 2010

Florida Remediation Studies Remedial and Developmental Postsecondary Courses  Lack of academic prep is a significant barrier to success –  Nationally approx. 40% of 1st year students are placed into college remediation (55-60% at CCs) How should this problem be addressed? Florida Remediation Studies Developmental courses at the community colleges All schools use the same placement instrument and have the same cutoff for placement Florida Administrative Dataset: Sample: All first- time, degree-seeking CC students who began in Fall 1997, 1998, 1999, or 2000 (over 100,000 records) Juan Carlos Calcagno and B.T. Long – "The Impact of Postsecondary Remediation Using a Regression Discontinuity Approach: Addressing Endogenous Sorting & Noncompliance“ and “Does Remediation Help All Students? The Heterogeneous Effects of Postsecondary Developmental Courses"

Regression Discontinuity: The Intuition Beth and Becky are observationally similar Beth and Becky both take the College Placement Test (CPT) Beth scores just above the cut-off score Becky scores just below the cut-off score Beth to college-level courses Becky to remediation Crossover: Beth takes remediation anyway Compare the outcomes of Beth & Becky No Show: Becky never enrolls in remediation => Explain setting! Then strategy 1 - COMPARE “SIMILAR” STUDENTS IS THE KEY 2 – DON’T DROP NON-COMPLETERS – BIAS RESULTS 3 – Limitation: students on the margin of the cutoff! Retesting: some students may be permitted to take the placement exam multiple times to pass out of the courses. Problem A: Re-taking might invalidates the key identifying assumption behind the RD design (i.e. students around the cutoff are “similar”). Results will be subject to selection bias. Problem B: No information on a student’s multiple test attempts; only the score used for placement is included Solution-Intuition: Identify colleges where retesting is allowed, and drop them from the sample. Mention cutoff – cutoff used for credit score applications I’ll use this exogenous force to solve the selection problem and predict outcomes, and not to look at the place of the cutoff – talk about this later Endogenous Sorting: Beth retests to place out of remediation

Negative Effects on Passing First College-Level Course Passing the First College-Level Course by Reading CPT Score at the Discontinuity Placed in Developmental Reading Placed in College Level Negative Effects on Passing First College-Level Course

Total Credits Earned by Reading CPT Score at the Discontinuity Placed in Developmental Reading Placed in College Level Positive Effects on Total Credits Earned

Florida Remediation Studies Being assigned to a developmental course appears to increase total credits completed for students on the margin of passing out of the requirement Does not increase the completion of college-level credits or eventual degree completion.  Remediation might promote early persistence but does not necessarily help students on the margin of passing the cutoff to make long-term progress What about students of different backgrounds? The effects were: More positive for female (compared to male), older (compared to younger), and LEP (compared to non-LEP) More negative for low-income student (compared to non-Pell-eligible)

Does remediation work for students with far less preparation? Past studies (FL, OH, TX) focus on students on the margin of needing remediation -- they do not investigate the effects of remediation on students who are extremely under-prepared (i.e. don’t have an appropriate control group) The Tennessee Case Multiple cutoffs and changes in placement policy over time  Investigate effects of different levels of remediation using multiple RDs Sample: Fulltime students under 21 who began at a TN public college or university in the fall of 2000 “Does Remediation Work for All Students? How Remedial and Developmental Courses Affect Students With Different Levels of Academic Preparation” – A. Boatman and B.T. Long

Remediation in Tennessee Go to College-level courses ACT/SAT score determines which placement exam (if any) Take Developmental Algebra II Take COMPASS Algebra exam Take Developmental Algebra I Take COMPASS Arithmetic exam Take Remedial Arithmetic College Level Devel. Alg II Devel. Alg I Remed. Arith. COMPASS Algebra II test Score 50-100 Score 28-49 Score 0-27 COMPASS Arithmetic test Score 30-100 Score 0-29 8

RD #1: Developmental Algebra II vs. College-level Devel. Alg II Devel. Alg I Remed. Arith. COMPASS Algebra II test Score 50-100 Score 28-49 Score 0-27 COMPASS Arithmetic test Score 30-100 Score 0-29

Credits Accumulated Over Three Years RD #1: Effects of Recommendation to Develop. Algebra II –vs– College-Level Math Credits Accumulated Over Three Years Not much Difference in Total Credits Diff in College Credits

Negative Effects in Cumulative College Credits RD #1: Effects of Recommendation to Develop. Algebra II –vs– College-Level Math Negative Effects in Cumulative College Credits

RD #3: Remedial Arithmetic vs. Develop Algebra I College Level Devel. Alg II Devel. Alg I Remed. Arith. COMPASS Algebra II test Score 50-100 Score 28-49 Score 0-27 COMPASS Arithmetic test Score 30-99 Score 0-29

Math Developmental/Remedial Education Total College Credits Completed after 3 years Smaller in Magnitude

College-Level vs. Developmental vs. Remedial Writing Composition Developmental Writing Remedial Writing COMPASS English test ACT English Score Score 68-100 Score 28-67 Score 0-27 RD #2: Develop. vs. Remedial Writing

Grade in first college-level course Not causal analysis, but interesting estimates…

Summary of Tennessee Results Generally negative effects if placed in a lower math class, but the size of the effect is smaller farther down the test score distribution Similar effects found for Developmental and Remedial Reading In Writing, the effects are negative at the college/developmental threshold but positive at the developmental/remedial threshold Positive effects on first college-level grade (though these results are not causal)

Implications and Remaining Questions Remedial and Developmental courses do not affect all types of students equally (by background, ability level, or college level) What is the best way to offer remediation? Characteristics of strong remedial programs? What are the effects of limitations states impose on remedial course-taking (e.g., only at CCs, time limits)? Early Placement Testing – a preventative measure?

Contact Information Prof. Bridget Terry Long, Ph.D. Harvard Graduate School of Education, NCPR, and NBER longbr@gse.harvard.edu Angela Boatman, Doctoral Candidate Harvard Graduate School of Education angela_boatman@mail.harvard.edu