1. Wisconsin Scholars Longitudinal Study: An Introduction SARA GOLDRICK-RAB & DOUGLAS N. HARRIS Co-Directors University of Wisconsin-Madison WWW.FINAIDSTUDY.ORG

2. Thanks for the Extensive Support! Funders: Bill and Melinda Gates Foundation, William T. Grant Foundation, Spencer Foundation Partners: University of Wisconsin System, Wisconsin Technical College System Board of directors of the Fund for Wisconsin Scholars Our staff, past, present, and future And especially our project manager, Alison Bowman

3. Policy Context The nation faces a college completion problem Completion rates are especially poor among students from low-income families Governments spend $155B+ on need-based financial aid each year (inc. grants & loans) To what degree is aid exerting an independent contribution to degree completion rates? Is it a cost-effective contribution? Through what mechanisms do effects arise? For which students is aid most effective?

4. How Financial Aid Could Help Students The conditions required to get the aid could provide motivation “I will continue to enroll in college in order to get this money.” “I will attend full-time rather than part-time in order to get this money.” The income could be used in positive or negative ways “This money helps me work a little less and study a little more.” “This money makes me less worried that I can get through college.” “With this money, I’ll stay in a dorm where I can hang out with my friends.”

5. Prior Research Studies suggest modest positive impacts of financial aid on college attendance and persistence for the average student The effects seem to vary depending on the conditions associated with the program They also depend on who the program is serving Isolating a causal effect of financial aid is particularly difficult

6. Grant vs. No Grant Comparison Ways to Evaluate the Effects of a Grant Program ddf 100 students sign up to receive a grant

7. Grant vs. No Grant Grant 100 students sign up No Grant1000s of students don’t sign up

8. Grant vs. No Grant Grant College Completion Rate= 70% No GrantCollege Completion Rate= 60%

9. Grant vs. No Grant Grant College Completion Rate= 70% No GrantCollege Completion Rate= 60% Students receiving the grants have higher college completion rates… Can we attribute that difference to the grants?

10. Scholarship vs. No Scholarship Scholarship College Completion Rate= 70% No ScholarshipCollege Completion Rate= 60% The observed differences in completion rates could be due to unobserved differences among students—which led them to get the grants.

11. Grant vs. No Grant This method compares apples to oranges The reason for not getting the grant could be associated with the reason for not finishing college (“selection bias”) If we know what the reason is, we can “control” for it If we don’t know what it is, or can’t observe it in the data we have, we can’t control for it The estimated impact of the grant mixes up these factors

12. Fund for Wisconsin Scholars A new, generous grant program Allocates grants with a method that facilitates rigorous estimation of impacts Willing to participate in an evaluation An uncommon opportunity to help large numbers of Wisconsin college students—while generating research knowledge needed to help students nationwide

13. Fund for Wisconsin Scholars Grants to students enrolled in UW System & the Wisconsin Technical College System Began making awards in fall 2008 Eligible students graduated from Wisconsin public high schools, enrolled in college within 3 years of graduation, for the first-time, and received a Pell Grant Had to have at least $1 of unmet need after all non- repayable sources of aid were accounted for Had to enroll full-time (12 credits) by the date of record

14. Fund for Wisconsin Scholars Grant amounts: $1,800 per year for students at 2-year colleges $3,500 per year for students at universities Terms of renewal: Continued receipt for up to 5 years Transferrable among publics Must register for 12 credits by the start of each new term Maintain ‘satisfactory academic progress’ (~C average) Terms are very comparable to the federal Pell Grant

15. Fund for Wisconsin Scholars Identification of eligible students: Fall of first year of college (after enrollment) FFWS works with Wisconsin’s financial aid officers Students don’t have to sign up for a “chance” to get the grants; they have already done the hard work of completing the FAFSA Selection process: Students are chosen (2-year and 4-year separately) Potential recipients are notified Students respond and verify their eligibility Award process: After verification, payments are disbursed via financial aid offices each term First payment arrives by 2 nd semester, year 1

16. Random Assignment of FFWS Grants   All eligible students have an equal chance of receiving a grant A coin toss helps ensure recipients and non- recipients are equivalent groups

17. FFWS Grant vs. No FFWS Grant This method compares apples to apples The reason for not getting the grant could NOT be associated with the reason for not finishing college The estimated impact of the FFWS grant is clean– if we observe differences in students’ outcomes they are due to the grant

18. How Students Get the FFWS Grant They are enrolled and already have financial aid The FFWS grant must go to the financial aid office & by law the student’s package must be adjusted if receiving the grant means their financial aid exceeds the institution’s cost of attendance Usually this means reducing loans and/work-study money; schools agreed not to reduce institutional aid So they get the grant as loan reduction, cash refund, or a combination, depending on the size and composition of their initial aid package

19. Wisconsin Scholars Longitudinal Study What are the average effects of the FFWS grant on college attainment? For which students is the grant most effective? Under what conditions is the grant most effective? (e.g. loan vs. cash, type of college) How does the FFWS grant affect students? How does it change how they spend their time? In what ways does it alter their relationships with other people? How does it affect how they think and feel?

20. Wisconsin Scholars Longitudinal Study Includes 3,000 students who were eligible for the FFWS grant in Fall 2008 1,200 students randomly chosen to receive the grant 1,800 students not chosen (at random) to receive the grant The sample is split between universities and 2- year colleges We also plan to include students who will be eligible for the FFWS grant in Fall 2012

21. Wisconsin Scholars Longitudinal Study We observe students’ outcomes with the following kinds of data: National Student Clearinghouse (enrollment anywhere) College transcripts Surveys (administered yearly, by mail) Interviews (conducted in-person, every semester, with a stratified sample of 36 students at 4 universities) We also plan to consider employment and earnings outcomes

22. Questions? About the study’s research questions? Design? Data?

23. Education Impacts of the Wisconsin Scholars Grant on University Students: 2008-2011 SARA GOLDRICK-RAB, DOUGLAS N. HARRIS, JAMES BENSON, & ROBERT KELCHEN University of Wisconsin-Madison WWW.FINAIDSTUDY.ORG

24. Sample 1,500 students who enrolled at one of Wisconsin’s 13 public universities in Fall 2008 All students in this analysis received a Pell Grant in that term We compare the outcomes of 600 students randomly selected to be offered the FFWS grant and 900 eligible students who were not chosen

25. Data Enrollment: National Student Clearinghouse (2008-2011) Obtained for all students Captures enrollment anywhere in the U.S., if NSC can find a “match.” Our “match” rate ~98%. Credits & GPA (2008-2010) Financial aid packages (2008-2010) Student surveys (2008, 2009) In-depth interviews (50 students, every 6 months, 2008- 11)

26. Were Students Offered and Not Offered the Grant Similar Before the Grant Was Awarded? Full SampleNon- Recipient Mean Treatment Difference P-Value % Female57.356.71.0.711 % Minority24.624.01.1.681 Average Age18.2 0.0.942 % First Gen53.453.5-0.2.958 % Dependent97.397.00.5.572 EFC ($)1,6331,60353.669 % $0 EFC30.631.9-2.3.362 Parent AGI ($)29,96329,4031,014.314 % 1st time in college95.995.70.5.681 Terms prior enroll1.8 -0.1.856 N1500900600

27. Question 1: How did the FFWS grant affect students’ financial resources?

28. Impact on Aid Package: Year 1 (2008- 2009) CONTROLTREATMENT IMPACT ($3,500) Total Aid$11,426 $1,665*** Pell (%)99.8 - 0.1 State Grant (%)99.0 - 3.7 *** SEOG (%)63.6 -9.8*** ACG (%)80.4 0.3 Institutional Aid (%)54.7 1.6 Work Study (%)18.3- 5.0 * Sub. Staff (%)77.9 - 11.1*** Unsub Staff (%)38.9 - 3.8 Total loans ($)3428.3 -909.5 93% of treatment group received FFWS this year. Aid amounts are unconditional on receipt—impacts reflect supplanting + student decisions

29. Impact on Aid Package: Year 2 (2009- 2010) CONTROLTREATMENT IMPACT ($3,500) Total Aid$10,082 $814.3 ** Pell (%) 74.3 - 0.6 State Grant (%) 67.2 0.5 SEOG (%)39.2 1.2 ACG (%)24.1 1.5 Institutional Aid (%) 48.1 -5.4 Work Study (%) 16.3 -3.6 Sub. Staff (%) 66.7 -7.2* Unsub Staff (%) 45.6 -9.8** Total loans ($)3581.2 -830.9 67% of treatment group received FFWS this year. Aid amounts are unconditional on receipt.

30. Summary About 1/3 of the amount of the FFWS grant was used to reduce students’ loans and work-study (as well as replace some state grant aid) After two years, students offered the FFWS grant had about $1,800 less debt than students not offered the grant

31. Question 2: How did the FFWS grant affect academic outcomes?

32. Average Impacts of the Grant on Enrollment: 2008-2011 Control MeanTreatment Effect Total # terms enrolled (f/s, %) 5.192 0.05 (.09) Ever enrolled summer (%)21.4 0.8 (2.6) Ever enrolled winter (%) 5.8 3.9 (1.7) ** Ever transferred (%)23.7 -0.4 (2.7) Ever attended 2-year college (%) 14.4 -0.4 (2.2) Completed associate’s degree (%) 3.1 0.4 (1.1) These are average effects, comparing FFWS recipients vs. control group Notes: *** p<.01, **p<.05, * p<.10

33. Average Impacts of the Grant on Credits: 2008-2010 Control MeanTreatment Effect Credit Accumulation Average credits completed46.9 0.9 (1.0) Earned 1-29 credits (%)18.2 0.4 (2.4) Earned 30-47 credits (%)16.0 2.6 (2.4) Earned 48-59 credits (%)42.3- 8.2 (3.1) *** Earned 60+ credits (%)22.2 6.3 (2.8) ** Progress toward 4-year Degree 60+ credits, 2-2.5 GPA2.2 0.0 (0.9) 60+ credits, 2.5-3.0 GPA6.8-0.5 (1.6) 60+ credits, 3.0-3.5 GPA8.5 4.7 (2.0) ** 60+ credits, 3.5+ GPA4.3 2.4 (1.4) * Notes: *** p<.01, **p<.05, * p<.10

34. Effects of the Grant Varied We compared the effects based on a student’s propensity to persist, using pre-FFWS factors: Age, parental education, race/ethnicity, gender, number of siblings Family income, assets, expected family contribution Years of high school math and science, GPA, ACT score, hours worked in high school, receipt of the Academic Competitive Grant Help from parents completing FAFSA # credits registered for 2 nd week of freshman year Receipt of the Supplemental Educational Opportunity Grant College attended

35. Propensity to Persist Overlapping disadvantages means that no single factor describes the category but parental education and ACT scores are some of the defining characteristics High (94%): Parents typically have at least a bachelor’s degree, students have higher than average test scores Middle (82%): Parents typically have a high school education, students have moderate test scores Bottom (55%): Parents typically have attended some college but not necessarily a university, students have lower test scores

36. Impact of the Grant on Enrollment by Propensity to Persist 2008-20092009-20102010-2011 Top: Control9994 Top: FFWS948379 Middle: Control 968582 Middle: FFWS 938481 Bottom: Control 946255 Bottom: FFWS 978372 -15 + 17

37. Impact of Grant on Total Number of Enrolled Terms: 2008-2011 Propensity to Persist

38. Impact of Grant on Number of Attempted Credits: 2009-2010 Propensity to Persist Note: No Impacts on Attempts In ‘08-09

39. Impact of Grant on Number of Completed Credits: 2008-2010 Propensity to Persist

40. Impact of Grant on Percent Completing 1-29 Credits: 2008-2010 Propensity to Persist

41. Impact of Grant on Percent Completing 60+ Credits: 2008-2010 Propensity to Persist

42. Summary On average, the grant appeared to accelerate time-to-degree for some students. This is promising since less than 30% of Pell recipients in Wisconsin complete bachelor’s degrees in 4 years. The grant appeared most effective for the students who were the least likely to persist in college

43. Are FFWS Requirements Driving the Effects? We think this is unlikely The requirements are not unusual Students think of the FFWS grant as simply as part of their aid package – it requires exactly what Pell does (12 credits, satisfactory academic progress) Many think the FFWS grant requires a B average– but they think the Wisconsin Higher Education Grant does too Awareness of the grant and its requirements is limited: In Fall 2008 survey just 37% of those awarded the grant reported getting it– a year later, just 49% of those awarded it seemed to know they had it

44. Decisions About Full-Time Enrollment Interviews do not indicate that students took specific grants into account when making decisions about the number of credits to take But they did consider overall financial constraints and the needs of their family members More important for determining the # credits was their perception of how well they were doing in school There was an overall lack of awareness of how number of credits relates to time-to-degree Lots of shifting from “4-year track” (15+ credits) to “5- year track” (12-14) credits, and sometimes to <12 too

45. Are FFWS Resources Driving the Effects? Resources could have many different types of effects For example, increased resources could allow students to substitute study time for work time The increased resources could also help students meet their other obligations, for example to family members, helping them stay enrolled The resources could prove necessary for continued enrollment, but insufficient to help them manage a higher credit load

46. Who Helped Students Make Decisions? Students who were unlikely to persist in college were first-generation students with close family ties: Their families may have worked extra hard to help them succeed in college, and they may have made decisions about college (including how to spend $) with their parents (Minikel-Lacocque & Goldrick-Rab, 2011) Students who were very likely to persist in college came from families where attending college was more normative, these were “emerging adults”: Students may have been afforded more independence about how to use the grant

47. Policy Implications Targeting of social programs can maximize their cost- effectiveness The challenge lies in identifying the best ways to target It can be difficult to find politically feasible approaches

48. Policy Implications Costs of the Pell Grant are substantial and growing (~$20-40B) The Obama administration wants to keep the grant maximum high while finding other places to cut One proposal is to require students to take 15 credits per term (instead of 12) to get the maximum Pell ($5,500) Only 31% of Pell recipients today take 15+ credits per term—most are juniors or seniors. This change means the Pell would be cut by $1000 or more for the 41% of Pell recipients currently taking 12-14 credits per term Assuming some of these succeeded in registering for 15+ credits, savings would be approximately $1-2 billion per year– more, if those at 12-14 credits drop down—or dropout Cost-effectiveness of the approach is unclear

49. Questions? Visit www.finaidstudy.org for a copy of the paperwww.finaidstudy.org