Download presentation
Presentation is loading. Please wait.
Published byBryanna Clerkin Modified over 9 years ago
1
***LILE and Aspirations question Reducing Barriers to Postsecondary Education Through Innovative Interventions: Evidence from Canadian Field Experiments McGill University Social Statistics Speaker Series March 21, 2012 Marc Frenette 1
2
Reducing Barriers to Postsecondary: The Need for Hard Evidence Improving access to PS important for many reasons: Private gains (Card, 1999; many others..) Societal gains, including productivity enhancement (Coulombe and Tremblay, 2006), crime reduction (Lochner and Moretti, 2004), longer life expectancy (Lleras-Muney, 2005), improved health (Arendt, 2008), and civic participation (Milligan, Moretti, and Oreopoulos, 2004) Equalizing opportunities / increasing intergenerational mobility 2
3
Reducing Barriers to Postsecondary: The Need for Hard Evidence The Canadian literature has successfully identified several important barriers to PS: Income (Frenette, 2008) Financial literacy (Frenette and Robson, 2011) Academic performance (Frenette, 2008) Career information (Frenette, 2009) 3
4
Reducing Barriers to Postsecondary: The Need for Hard Evidence Although all four barriers are amenable to policy intervention, we have little to no hard evidence on best practices in Canada: Students who qualify for grants or participate in academic / career counseling are likely a select group Lack of natural experiments in Canada Field experiments, using random assignment, are ideal, but rare… 4
5
The Canadian Millennium Scholarship Foundation Recognized the Need for Hard Evidence (and they had hard cash!!!) Commissioned the Social Research and Demonstration Corporation (SRDC) to design, implement, and evaluate three interventions to reduce specific barriers to PSE: BC AVID –Academic intervention for middle-achieving students Learning Accounts (Future to Discover) –Early promise of a large non-repayable grant for low-income students Explore Your Horizons (Future to Discover) –Career / education information 5
6
AVID Advancement Via Individual Determination (AVID) Serves 400,000 students in 4,500 schools worldwide Goal: raise academic performance, and subsequently, university attendance Targets students on the margin Mostly Bs and Cs (middle-achieving) No extreme behaviour issues Desire to attend university 6
7
AVID What is AVID? Academic intervention Elective class (replacing a regular elective) offered throughout high school Consists of: –Curriculum studies (40%) –Tutorials assisted by local college students (40%) –Motivational activities regarding PS (20%) 7
8
AVID Mechanisms through which AVID may help students (Dunn et al., 2008)? Study skills (time management, note taking, etc.) “Untracking” students (students choosing advanced courses) Mentoring effects (continued contact with AVID teacher and tutors) Peer effects (continued contact with students sharing similar characteristics) 8
9
AVID Can AVID make a difference? Cognitive skills are not malleable after age 14 (Heckman, 1995), when AVID begins –But AVID helps students use their existing cognitive skills more efficiently by helping them become better learners Furthermore, non-cognitive skills (motivation, self-discipline) are controlled by the prefrontal cortex, which is malleable until late adolescence (Heckman, 2000; Shonkoff and Phillips, 2000) 9
10
AVID Non-experimental evidence on AVID’s effectiveness: Mehan et al. (1996): –Improvements in college participation Watt et al. (2006) –AVID districts in Texas saw gains in graduation rates, advanced course enrollments, and international baccalaureate testing Watt et al. (2007) –AVID participants had higher aspirations, knowledge about college, and academic preparation compared to peers Nagaoka and LaForce (2010) –Propensity score matching study in Chicago –Small improvements in English and Math in grade 9; fewer absences All suffer from selection bias Those most likely to benefit from AVID will sign up 10
11
BC AVID First (and only) scientific evaluation of AVID 1,241 students recruited at 14 BC high schools* Two cohorts (2005 and 2006) followed throughout HS and until university age Administrative and survey data used for follow-up ‘AVID eligible’ students recruited through rigorous process: Middle-achieving (mostly Bs and Cs) No extreme behaviour issues Desire to attend university *Schools and students had to apply to participate and waiting lists were created when classes reached the limit of 30 (just like the real AVID). Informed consent was also required from the parents to collect data. 11
12
Evaluation approach Random assignment within high schools Impact: Treatment group outcome – Control group outcome Controlling for baseline characteristics 12
13
Focus of this study Impact of the offer of AVID on highest level of education aspired*: Early demand indicator Future work will report on university / college application / attendance 13 * “No matter what you plan to be doing in a year from now (grade 12), what is the highest level of education you would like to get?”
14
Large impact on university aspirations for boys and first-generation students 14 Weaker AVID skills to begin with?
15
No impact on non-university PS, as expected 15
16
AVID may need more time to work… 16 -
17
Potential biases/evaluation challenges Those we can rule out Program take-up 96.7% accepted AVID offer Spillover of AVID techniques Very minor issue based on earlier comparison with similar non-AVID schools (Cornell notes) Attrition bias (survey data analysis only) 19.6% (treatment) vs 23.8% (control) between grade 9 and 12 No important changes in baseline characteristics Teacher grading bias AVID students are well known Argument does not apply to differences in sub-group impacts 17
18
Potential biases/evaluation challenges Those we can not rule out Program drop-out 37.6% withdrew by grade 12 (mostly in that year: wanted to pick other courses to apply skills) Estimated impacts = Intention-to-treat effects Substitution bias (displacing non-experimental treatment) Earlier report found reverse: course load was more challenging among treatment group Timing Lagged effect Students coped with more difficult course load as they were only beginning to learn AVID skills AVID now being run before HS in some jurisdictions 18
19
Potential biases/evaluation challenges Those we can not rule out Reactions to inequity caused by randomization Positive or negative impact on control group Currently following outcomes of similar students in matched non-AVID schools Sample size Affects statistical significance (especially in sub-group analysis) 19
20
BC AVID Final Report Final report (including cost-benefit study) in 2013 20
21
Future to Discover Overview Two interventions designed to increased PSE access: Learning Accounts –Early promise of non-repayable grant (up to $8,000) for low-income youth –Offered in NB Explore Your Horizons –Career and PSE planning intervention –Several components (workshops, magazines, parental outreach, etc.) –Offered in NB and MB 21
22
How Can Learning Accounts Help? Reduces cost of post-secondary, like other grants (clawed- back from loans) Potentially addresses several design issues with existing grants: –Students are informed of the grant –Early promise of aid –No obligation to apply for loans oMay encourage loan averse students to seek grant Palameta and Voyer (2010) Lab experiment: 5-20% students rejected grants when couple with a loan offer, but accepted same grant without loan offer (even though loan could be forfeited without compromising grant) oWith no loans, Learning Accounts represents additional aid 22
23
How Can Explore Your Horizons Help? Informs students of costs and benefits of PSE Financial literacy of students very poor regarding PSE (Frenette and Robson, 2011) Helps students understand educational requirements for the careers they wish to pursue Knowledge of educational requirements an issue for many students (Frenette, 2009) 23
24
Methodology Students are randomly assigned to receive the intervention (treatment group) or not (control group) Three possible interventions: LA EYH LA + EYH 24
25
25 No PSE credential
26
26
27
Why did Learning Accounts work in Francophone sector? Possible explanation: supply constraints Data from New Brunswick government: Anglophone programs were oversubscribed Francophone programs were undersubscribed 27
28
28
29
29
30
30
31
31
32
32
33
Potential biases/evaluation challenges Program take-up/drop-out Everyone received offer of LA, but about 30% did not remember a couple of years later Attendance in EYH workshops was poor, especially in Manitoba –Non-mandatory, after school Spillover effects Not an issue for LA EYH workshops held after class Substitution bias (displacing non-experimental treatment) Minor impact in LA (small reduction in non-repayable aid for some groups) EYH: looking into it Attrition bias Enrolment numbers use administrative data 33
34
Potential biases/evaluation challenges Reactions to inequity caused by randomization Positive or negative impact on control group No mechanism for testing this (did not follow non-experimental schools) Sample size Affects statistical significance (especially in sub-group analysis) 34
35
Future to Discover Final Report Final report (including cost-benefit study) in September 2012 35
36
Appendix: Related academic interventions QOP (Hahn et al., 1994; Rodriguez-Planas, 2010) Includes financial incentives IHAD (Kahne and Bailey, 1999) Includes early promise of ‘last dollar scholarship’ Career Academies (Kemple and Willner, 2008) Focused on career themes GEAR UP Upward Bound (Seftor et al., 2009) Upward Bound Math-Science (Seftor and Calcagno, 2010) 36 Treatment broadly similar to AVID, but focus on low-income students
Similar presentations
© 2024 SlidePlayer.com Inc.
All rights reserved.