1. The Impact of Postsecondary Remediation Using a Regression Discontinuity Approach: Addressing Endogenous Sorting and Noncompliance Juan Carlos Calcagno Mathematica Policy Research, Inc. IES Research Conference June 11 th, 2008

2. Juan Carlos Calcagno, June 11th, 2008 2 :: 12 >> Introduction: Why is this Important?  Little is known about the causal effects of remediation on student outcomes: »Lack of data »Nonrandom selection into remediation  Controversial topic in postsecondary education: »Remediation increases access but is costly to states and colleges »Should we pay twice for academic preparation that should have occurred in secondary school? »Large proportion of entering students requiring remedial courses »Remedial credits do not count toward degrees, but students pay tuition for these courses

3. Juan Carlos Calcagno, June 11th, 2008 3 :: 12  Estimate the causal impact of remediation in Florida community colleges using a unique administrative dataset.  Use a quasi-experimental design (regression discontinuity) based on a remedial placement cutoff rule to solve the selection problem.  Discuss threats to internal validity and implications for practice: noncompliance and endogenous sorting >> Contribution of this Study

4. Juan Carlos Calcagno, June 11th, 2008 4 :: 12 >> FL Administrative Dataset  All first-time, degree-seeking community college students starting in the Fall of 1997 to 2000 (over 100,000 records)  Term-by-term transcript data through Spring of 2006  All test scores reported (CPT, SAT, ACT) plus other controls  Outcomes: passing Algebra/English 101, retention, certificate, associate degree, transfer to state 4-year college, credits earned (remedial and non-remedial)

5. Juan Carlos Calcagno, June 11th, 2008 5 :: 12 >> The Regression Discontinuity Design: Intuition Beth and Becky are observationally similar Compare the outcomes of Beth and Becky Beth and Becky both take the College Placement Test (CPT) Beth scores just above the cut-off score Becky scores just below the cut-off score Crossover: Beth take remediation No-show: Becky never enroll in remediation Beth go to college-level coursesBecky go to remediation Endogenous Sorting: Beth retests to place out of remediation

6. Juan Carlos Calcagno, June 11th, 2008 6 :: 12 >> Noncompliance Probability of Enrollment in Remediation by CPT Score and Subject ` CPT Score Relative to Math CutoffCPT Score Relative to Reading Cutoff

7. Juan Carlos Calcagno, June 11th, 2008 7 :: 12 >> Endogenous Sorting around Cutoff Density of Reading CPT for Institution E 0.01.02.03.04 -50-40-30-20-100102030

8. Juan Carlos Calcagno, June 11th, 2008 8 :: 12 >> The Econometric Model D: assignment to remediation (below cutoff) is the intention-to-treat estimator (ITT) Z: CPT test score Robustness tests: (1) add controls; (2) narrow sample; (3) endogenous sorting; (4) functional form & standard errors is the local average treatment effect (RD-IV) T: enrollment in remedial education

9. Juan Carlos Calcagno, June 11th, 2008 9 :: 12 >> The Impact of Math Remediation

10. Juan Carlos Calcagno, June 11th, 2008 10 :: 12 >> The Impact of Reading Remediation

11. Juan Carlos Calcagno, June 11th, 2008 11 :: 12 >> Summary of Findings  Math remediation increases persistence to 2nd year (2 to 3.8%) for students on the margin of passing the cutoff.  Impacts on total credits earned are positive (7 and 3 points, M & R), but not different from zero for college-level credits.  No statistical differences for all other outcomes  The likelihood of passing English 101 was slightly lower for reading remedial students while no difference was found in passing Algebra 101 for math remedial students.

12. Juan Carlos Calcagno, June 11th, 2008 12 :: 12 >> Policy Implications  Remediation might promote early persistence in college, but it does not necessarily help students on the margin of passing the cutoff to make progress toward a degree.  Costs of remediation should be given careful consideration in light of the limited benefits.  State Departments of Education should explore noncompliance and retesting practices and consider potential consequences.