Presentation is loading. Please wait.

Presentation is loading. Please wait.

***LILE and Aspirations question Reducing Barriers to Postsecondary Education Through Innovative Interventions: Evidence from Canadian Field Experiments.

Similar presentations


Presentation on theme: "***LILE and Aspirations question Reducing Barriers to Postsecondary Education Through Innovative Interventions: Evidence from Canadian Field Experiments."— Presentation transcript:

1 ***LILE and Aspirations question Reducing Barriers to Postsecondary Education Through Innovative Interventions: Evidence from Canadian Field Experiments McGill University Social Statistics Speaker Series March 21, 2012 Marc Frenette 1

2 Reducing Barriers to Postsecondary: The Need for Hard Evidence  Improving access to PS important for many reasons: Private gains (Card, 1999; many others..) Societal gains, including productivity enhancement (Coulombe and Tremblay, 2006), crime reduction (Lochner and Moretti, 2004), longer life expectancy (Lleras-Muney, 2005), improved health (Arendt, 2008), and civic participation (Milligan, Moretti, and Oreopoulos, 2004) Equalizing opportunities / increasing intergenerational mobility 2

3 Reducing Barriers to Postsecondary: The Need for Hard Evidence  The Canadian literature has successfully identified several important barriers to PS: Income (Frenette, 2008) Financial literacy (Frenette and Robson, 2011) Academic performance (Frenette, 2008) Career information (Frenette, 2009) 3

4 Reducing Barriers to Postsecondary: The Need for Hard Evidence  Although all four barriers are amenable to policy intervention, we have little to no hard evidence on best practices in Canada: Students who qualify for grants or participate in academic / career counseling are likely a select group Lack of natural experiments in Canada Field experiments, using random assignment, are ideal, but rare… 4

5 The Canadian Millennium Scholarship Foundation Recognized the Need for Hard Evidence (and they had hard cash!!!)  Commissioned the Social Research and Demonstration Corporation (SRDC) to design, implement, and evaluate three interventions to reduce specific barriers to PSE: BC AVID –Academic intervention for middle-achieving students Learning Accounts (Future to Discover) –Early promise of a large non-repayable grant for low-income students Explore Your Horizons (Future to Discover) –Career / education information 5

6 AVID  Advancement Via Individual Determination (AVID)  Serves 400,000 students in 4,500 schools worldwide  Goal: raise academic performance, and subsequently, university attendance  Targets students on the margin Mostly Bs and Cs (middle-achieving) No extreme behaviour issues Desire to attend university 6

7 AVID  What is AVID? Academic intervention Elective class (replacing a regular elective) offered throughout high school Consists of: –Curriculum studies (40%) –Tutorials assisted by local college students (40%) –Motivational activities regarding PS (20%) 7

8 AVID  Mechanisms through which AVID may help students (Dunn et al., 2008)? Study skills (time management, note taking, etc.) “Untracking” students (students choosing advanced courses) Mentoring effects (continued contact with AVID teacher and tutors) Peer effects (continued contact with students sharing similar characteristics) 8

9 AVID  Can AVID make a difference? Cognitive skills are not malleable after age 14 (Heckman, 1995), when AVID begins –But AVID helps students use their existing cognitive skills more efficiently by helping them become better learners Furthermore, non-cognitive skills (motivation, self-discipline) are controlled by the prefrontal cortex, which is malleable until late adolescence (Heckman, 2000; Shonkoff and Phillips, 2000) 9

10 AVID  Non-experimental evidence on AVID’s effectiveness: Mehan et al. (1996): –Improvements in college participation Watt et al. (2006) –AVID districts in Texas saw gains in graduation rates, advanced course enrollments, and international baccalaureate testing Watt et al. (2007) –AVID participants had higher aspirations, knowledge about college, and academic preparation compared to peers Nagaoka and LaForce (2010) –Propensity score matching study in Chicago –Small improvements in English and Math in grade 9; fewer absences  All suffer from selection bias Those most likely to benefit from AVID will sign up 10

11 BC AVID  First (and only) scientific evaluation of AVID  1,241 students recruited at 14 BC high schools* Two cohorts (2005 and 2006) followed throughout HS and until university age Administrative and survey data used for follow-up  ‘AVID eligible’ students recruited through rigorous process: Middle-achieving (mostly Bs and Cs) No extreme behaviour issues Desire to attend university *Schools and students had to apply to participate and waiting lists were created when classes reached the limit of 30 (just like the real AVID). Informed consent was also required from the parents to collect data. 11

12 Evaluation approach  Random assignment within high schools  Impact: Treatment group outcome – Control group outcome Controlling for baseline characteristics 12

13 Focus of this study  Impact of the offer of AVID on highest level of education aspired*: Early demand indicator  Future work will report on university / college application / attendance 13 * “No matter what you plan to be doing in a year from now (grade 12), what is the highest level of education you would like to get?”

14 Large impact on university aspirations for boys and first-generation students 14 Weaker AVID skills to begin with?

15 No impact on non-university PS, as expected 15

16 AVID may need more time to work… 16 -

17 Potential biases/evaluation challenges Those we can rule out  Program take-up 96.7% accepted AVID offer  Spillover of AVID techniques Very minor issue based on earlier comparison with similar non-AVID schools (Cornell notes)  Attrition bias (survey data analysis only) 19.6% (treatment) vs 23.8% (control) between grade 9 and 12 No important changes in baseline characteristics  Teacher grading bias AVID students are well known Argument does not apply to differences in sub-group impacts 17

18 Potential biases/evaluation challenges Those we can not rule out  Program drop-out 37.6% withdrew by grade 12 (mostly in that year: wanted to pick other courses to apply skills) Estimated impacts = Intention-to-treat effects  Substitution bias (displacing non-experimental treatment) Earlier report found reverse: course load was more challenging among treatment group  Timing Lagged effect Students coped with more difficult course load as they were only beginning to learn AVID skills AVID now being run before HS in some jurisdictions 18

19 Potential biases/evaluation challenges Those we can not rule out  Reactions to inequity caused by randomization Positive or negative impact on control group Currently following outcomes of similar students in matched non-AVID schools  Sample size Affects statistical significance (especially in sub-group analysis) 19

20 BC AVID Final Report  Final report (including cost-benefit study) in

21 Future to Discover Overview  Two interventions designed to increased PSE access: Learning Accounts –Early promise of non-repayable grant (up to $8,000) for low-income youth –Offered in NB Explore Your Horizons –Career and PSE planning intervention –Several components (workshops, magazines, parental outreach, etc.) –Offered in NB and MB 21

22 How Can Learning Accounts Help?  Reduces cost of post-secondary, like other grants (clawed- back from loans)  Potentially addresses several design issues with existing grants: –Students are informed of the grant –Early promise of aid –No obligation to apply for loans oMay encourage loan averse students to seek grant  Palameta and Voyer (2010)  Lab experiment: 5-20% students rejected grants when couple with a loan offer, but accepted same grant without loan offer (even though loan could be forfeited without compromising grant) oWith no loans, Learning Accounts represents additional aid 22

23 How Can Explore Your Horizons Help?  Informs students of costs and benefits of PSE Financial literacy of students very poor regarding PSE (Frenette and Robson, 2011)  Helps students understand educational requirements for the careers they wish to pursue Knowledge of educational requirements an issue for many students (Frenette, 2009) 23

24 Methodology  Students are randomly assigned to receive the intervention (treatment group) or not (control group)  Three possible interventions: LA EYH LA + EYH 24

25 25 No PSE credential

26 26

27 Why did Learning Accounts work in Francophone sector?  Possible explanation: supply constraints  Data from New Brunswick government: Anglophone programs were oversubscribed Francophone programs were undersubscribed 27

28 28

29 29

30 30

31 31

32 32

33 Potential biases/evaluation challenges  Program take-up/drop-out Everyone received offer of LA, but about 30% did not remember a couple of years later Attendance in EYH workshops was poor, especially in Manitoba –Non-mandatory, after school  Spillover effects Not an issue for LA EYH workshops held after class  Substitution bias (displacing non-experimental treatment) Minor impact in LA (small reduction in non-repayable aid for some groups) EYH: looking into it  Attrition bias Enrolment numbers use administrative data 33

34 Potential biases/evaluation challenges  Reactions to inequity caused by randomization Positive or negative impact on control group No mechanism for testing this (did not follow non-experimental schools)  Sample size Affects statistical significance (especially in sub-group analysis) 34

35 Future to Discover Final Report  Final report (including cost-benefit study) in September

36 Appendix: Related academic interventions  QOP (Hahn et al., 1994; Rodriguez-Planas, 2010) Includes financial incentives  IHAD (Kahne and Bailey, 1999) Includes early promise of ‘last dollar scholarship’  Career Academies (Kemple and Willner, 2008) Focused on career themes  GEAR UP  Upward Bound (Seftor et al., 2009)  Upward Bound Math-Science (Seftor and Calcagno, 2010) 36 Treatment broadly similar to AVID, but focus on low-income students


Download ppt "***LILE and Aspirations question Reducing Barriers to Postsecondary Education Through Innovative Interventions: Evidence from Canadian Field Experiments."

Similar presentations


Ads by Google