Presentation on theme: "Presentation Outline The Future of Institutional Research (IR) & Technology in improving first-year students’ success. Example 1: Demonstration of an."— Presentation transcript:
Presentation Outline The Future of Institutional Research (IR) & Technology in improving first-year students’ success. Example 1: Demonstration of an IR innovation. Example 2: Demonstration of a Technology innovation.
The Future of IR and Technology IR’s future is moving beyond reporting to analysis. This means converting data into ‘actionable’ information that FYE personnel can use. Technology’s future is moving beyond data management to production of tools that directly facilitate and improve student success.
Example 1: Student-at-Risk Prediction Model Also known as a predictive model, or enrollment forecasting model. Helps answer questions like: – Which student variables are most useful for predicting freshmen retention? – What is the “best” combination of variables to optimize predictions? – How useful is this combination for identifying at-risk students?
Relevant Previous Research Astin, A. W. (1993). What matters in college? Four critical years revisited. San Francisco: Jossey-Bass. Bean, J. P. (1985). Interaction effects based on class level in an explanatory model of college student dropout syndrome. American Educational Research Journal, 22(1), 35–64. Caison, A. L. (2006). Analysis of institutionally specific retention research: A comparison between survey and institutional database methods. Research in Higher Education, 48(4), 435-451. Herzog, S. (2006). Estimating student retention and degree-completion time. Decision trees and neural networks vis-à-vis regression. New Directions for Institutional Research, 131, 17-33. Pascarella, E., and Terenzini, P. (2005). How College Affects Student: Volume 2, A Third Decade of Research. San Francisco: Jossey-Bass. Sujitparapitaya, S. (2006). Considering student mobility in retention outcomes. New Directions for Institutional Research, 131, 35-51. Tinto, V. (1975). Dropout from higher education: A theoretical synthesis of recent research. Review of Educational Research, 45(1), 89-125.
4 Steps to Modeling Retention 1. Get Freshmen Data. (i.e. We used fall 2009 & 2010 data to build our “training” data set.) 4. Check the actual 2011 retention outcomes to see how well the model performed. 2. Build Model. 3. Apply model parameters to new data. (i.e. model validation, scoring) RETENTION
Examples of Student Variables Analyzed Gender Age Ethnicity Residency Geographic Origin High School GPA & Rank SAT AP CLEP Educational Goals Transfer GPA # Transfer Credits Major Credit Load Credits Earned First Term GPA Distance Education Dual Enrollment High Failure Rate Courses Courses Taken (including Math & English) On Campus Employment Housing Student Life Activities Athletics STAR Usage Average Class Size Need Based Aid Non-need Based Aid Pell Grant Work Study % of Aid Met Ethnicity by Geographic Origin Employment by Housing High School GPA by First Term GPA Residency by Need Based Aid Ratio of Successful Adds to Drops PERSISTENCE DEMOGRAPHICS ACADEMIC CAMPUS EXPERIENCE FINANCIAL NEED INTERACTIONS Credits earned Credits attempted Credit Completion Ratio Math/English Enrollment/Completion Continuous Enrollment Milestone metrics MILE- STONES PRE- COLLEGE
Continental US High School GPA % Need Met Educational Goals AP/CLEP Credit FYE Class15 Credits On Campus Work RETENTION IN YEAR 1 Strongest Weakest These variables account for approximately 39% of the variance in a student’s likelihood of returning for a third semester (Pseudo R Square =.387). *Wald statistic (sig.) The Wald test statistic was used to indicate strength of the variable instead of the coefficient, standardized beta. Because of the nature of the logistic regression, the coefficient is not easily interpretable to indicate strength. 262.804 (.000)* 101.368 (.000)* 7.817 (.005)* 20.292 (.000)* 13.486 (.000)* 7.134 (.008)* 4.419 (.036)* 3.791 (.052)* 7 Strongest Predictors of Retention
Predictors in Regression Equation BS.E.WalddfSig.Exp(B) Step 1 a ED GOALS.575.15713.4861.0001.778 HS GPA1.761.175101.3681.0005.817 CONTINENTAL US -2.544.157262.8041.000.079 FYE CLASS.393.1477.1341.0081.481 FIN NEED MET1.021.22720.2921.0002.777 ON CAMPUS WORK.411.2113.7911.0521.508 FIFTEEN CREDITS.267.1274.4191.0361.306 AP/CLEP.453.1627.8171.0051.573 Constant-6.623.628111.2001.000.001 a.Variable(s) entered on step 1: EDGOALS, HSGPA, MAINLAND, CAS110, FINNEED, EMPLOY, FIFTEENCREDITS, APCLEP. Pseudo Rsquare =.387
Scoring of relative dropout/retention risk p = exp (a+b 1 x 1 +b 2 x 2 +b 3 x 3 +b 4 x 4 ….) 1 + exp (a+b 1 x 1 +b 2 x 2 +b 3 x 3 +b 4 x 4 ….) Where:p = probability of enrollment/non-enrollment exp = base of natural logarithms (~ 2.72) a = constant/intercept of the equation b = coefficient of predictors (parameter estimates) Scoring Students
John: – is from the continental U.S. (0) – has a below average high school GPA (2.65) – is enrolled in 9 credits (9) – has a low % of financial need met (.45) – isn’t not working on campus (0) – isn’t enrolled in CAS 110 (0) – didn’t specify any educational goals in survey (0) Probability of Dropping: 0.77 Example: John is at risk of dropping
Sample Data for FYE Advisors UH IDAGE GENDERETHNICITY COLLEGEDEPTMAJOR Ed Goal Specified Relative Risk Value Risk Level 00118 FCH CA&HARTBAYes14.92LOW 00218 FHW CSSSOCBAYes36.88MEDIUM 00319 MAA CENGEEBSNo89.18HIGH UH ID LAST NAME FIRST NAME EMAIL CURRENT CREDITS RESIDENT AP/ CLEP HS GP A WORK ON CAMP 1 st YR EXP CLASS % FIN NEED MET STAR LOGINS ADVISOR PREVIOUS CONTACT 001 12HI63.80YY77%0Y 002 15HI03.13NN43%3N 003 16CA62.59YY65%2N
407 freshmen from 2011 dropped out in year one. Retaining just 22 students from 2011 would have improved Mānoa’s overall retention rate from 78.8% to 80%. Additional Revenue from Tuition and Fees = $210,000 (for 16 HI, 6 WUE, excludes out-of-state!). Are there 22 students in this group that we can help/retain? Impact on Campus
Gary Rodwell Director of Advanced Technology & Lead Architect of ‘STAR’ University of Hawaii at Manoa Example 2: ‘STAR’ Technology
Reed Dasenbrock Vice Chancellor for Academic Affairs John Stanley Institutional Analyst Gary Rodwell Director of Advanced Technology University of Hawaii at Manoa Questions: firstname.lastname@example.org@hawaii.eduMahalo