All material in this presentation, including text and images, is the property of Noel-Levitz, Inc. Permission is required to reproduce information. How can IR help with assessment and retention? A presentation for the Oklahoma Association for Institutional Research and Planning by Scott E. Bodfish Vice President May 5, 2009

Today’s Agenda Outline of a process for working with campus retention and assessment initiatives Case studies to illustrate that process Presentation of some strategies for effective research NOTE: Please mute your phone –*6 to mute; #6 to unmute

So, your provost ordered the raw data from your satisfaction/priorities survey

Here is what she got!

Now she wants you to tell her what it all means

Although it will be pleasant you can waste a lot of time analyzing the data without some kind of plan

The Research Design Process Define the problem Develop a research objective Collect data Turn data into information Get information to decision-makers

The Diagnostic Process There is a problem (symptoms) The symptoms have an underlying cause The nature of the cause allows for intervention Implementing the intervention

Example The number of students failing introductory statistics courses has increased. Many of them have lower scores on basic math skills (specifically algebra) than previous cohorts of students. We can train advisors to counsel students into remedial math based on assessment test score. Administer assessment tests; train advisors; monitor statistics course pass rates.

Developing targeted counseling services at a four-year public university Counseling and advising of new students doesn’t seem to be working well All students are going through the same process; no customization We need a means for assessing student needs that allow for customization Identify an instrument; administer and prioritize services accordingly

The situation Program serves at-risk students (economically disadvantaged; inadequate academic preparation) Sought an instrument to assess student needs for intervention and support beyond academic remediation CSI not only identifies needs but assesses receptivity to help

Findings from the research 20% of entering freshmen had plans to work hours per week –Include two time management seminars in our summer programming –Counselors know to specifically speak with students about work and the time required to be successful in their studies

Findings from the research 46% of the initial respondents indicated that the highest degree they sought would be a graduate degree –A workshop on graduate studies was introduced for students in their second year rather than at the end of the third year

Findings from the research Women more receptive to academic assistance and personal counseling while men more receptive to career counseling –Counselors structure the topics differently for men than for women; however –Individuals vary within each gender group so the primary focus is on each individual’s specific receptivity

Outcomes Now provide in-classroom supplemental instruction for 2 levels of math for students identified with this need Hired two academic coaches to assist students who scored high on motivation and receptivity scales, but low on confidence and study skills An increase in the “show rate” to 87% for all booked coaching sessions

Improving facilities and services at a community college Rapid enrollment growth and general complaints about student service / satisfaction No way of prioritizing student service initiatives Identify and prioritize specific concerns Ongoing measurement of levels of satisfaction

The situation Selected SSI over “home grown” instrument in order to use a standardized instrument used for self-study Have administered SSI every other year since 1996 Administration procedures have remained consistent over time

Use of Data to Make Changes Created new library (information commons) in late 1990’s –Satisfaction w/ library services increased from 5.22 to 5.64 Created new student life center –Satisfaction (it is enjoyable to be a student on this campus) increased from 4.72 to 5.41

Use of Data to Make Changes Consolidated counseling services –Slight increase in satisfaction with advising Incorporated student development services (admissions, financial aid, registration, career services, disability services) along w/ counseling –Much greater increase in satisfaction Changed course scheduling process –Second largest increase in satisfaction

Use of Data to Make Changes Added parking in 1998 –Satisfaction increases from 3.35 to 3.81 Enrollment increases –Satisfaction decreases to 3.54 Added more parking in 2002 –Satisfaction increases to 4.31 Enrollment has increased again –Satisfaction has declined to 3.75

Understanding attrition at a four-year private university An increase in attrition rates Engagement and academic performance (not finances) related to withdrawal Change marketing / recruiting messages to create more accurate expectations Ongoing measurement of attrition

The plan before research Private, four-year institution (high cost) Anecdotal evidence that students feel they aren’t receiving enough financial aid We should increase institutional financial aid packages for continuing students

Financial considerations the most common reason for leaving college SOURCE: ELS:2002 “A First Look at the Initial Postsecondary Experiences of the High School Sophomore Class of 2002 (National Center for Education Statistics)

The research project Structured interviews with 20 students (10 who persisted and 10 who withdrew) Analysis of institutional historical data files (two cohorts) –Data included EFC, financial aid awards, admissions data (HS GPA, SAT scores), academic performance data (credits registered, credits completed, GPA earned)

Findings from the research Some students, especially freshmen, seem to have a low initial commitment to the university. Academic performance in the first semester sets the tone for most students. Financial need is unrelated to whether or not a student will withdraw Basic measures such as high school GPA and class rank provide a better indication of which students will have difficulty academically

So what about that survey data the provost requested?

Why focus on student satisfaction? Schools with higher overall satisfaction enjoy: –Higher graduation rates –Lower loan default rates –Higher alumni giving rates

Identifying the “drivers” of satisfaction

What other variables influence a student’s college experience?

What are some strategies for working effectively with campus retention or assessment initiatives?

Be clear on institutional priorities

There are regular tasks that aren’t strategic but still high priority

Cultivate a relationship with the retention or assessment team

Brainstorm with them to get their ideas

Make joint requests for resources

Share the successes

Relate specific results to insights from the data

What kind of IR / Planning shop do you have?

We just fill out institutional surveys

Important to have consistent institutional data reported

There are more surveys than there is time available

You may be able to “mine” the surveys for data you need

We just manage surveys or extract data from internal databases

Is it time to re-visit the survey calendar in light of retention or assessment initiatives?

Are there regular reports that provide information for retention or assessment?

The “internal consultant”

Tends to be entrepreneurial

More often reports to the president than to a vice president

May have a specialized focus or interest

There are always more data to analyze, more studies to design and more reports to write

Having a process that begins with defining the problem will help clear the way