1 Tracking College Students Over Time: Using CIRP & CSS Data for Longitudinal Assessment Victor Sáenz (UCLA) John Pryor (UCLA) & Gavin Henning (Univ. of New Hampshire) Wednesday, May 17th, 2006 AIR Forum Chicago, IL

2 Presentation Goals Overview of CIRP Freshman Survey Introduce the CSS survey as a longitudinal follow-up to the CIRP Freshman Survey Discuss how institutions can utilize HERI longitudinal data to assess various aspects of the first-year experience Share descriptive findings from the 2005 CSS national aggregate data Institutional perspective on using HERI longitudinal data

3 Higher Education Research Institute CIRP Cooperative Institutional Research Program Freshman Survey YFCYCSS Faculty Survey Funded Research Atlantic Philanthropies Templeton Foundation National Institutes of Health

4 CIRP, CSS, & the I-E-O Model Inputs CIRP Freshman Survey (e.g., academic performance in high school, financial concerns prior to college entry, expectations for college, degree aspirations, self-concept in high school) Environments CSS (e.g., place of residence during college, interactions with peers and faculty, curricular and co-curricular experiences) Outcomes CSS (e.g., satisfaction with college, retention, post-college plans)

5 Using CIRP & CSS Data to Enhance Campus Assessment Efforts: Methodologies Descriptive analyses with your campus data Comparative analyses Measures of association Longitudinal analyses Factor analyses Multivariate analyses

6 Using CIRP & CSS Data to Enhance Campus Assessment Efforts: Purposes Self-study reports Retention studies Recruitment issues Examining group differences among students Strategic planning Presentations to various constituents Creation of a student information system Measuring student development & institutional impact

7 CIRP Freshman Survey Largest and longest-running national study of American college students…2006 is 40 th Anniversary Initiated in 1966 at the American Council on Education; Housed at HERI (UCLA) since 1973 Administered annually to over 400,000 incoming freshmen at more than 600 colleges and universities nationwide Since 1966: 12 million students; 1,800 institutions

8 CIRP Freshman Survey Items Demographic Characteristics Expectations for College High School Experiences Degree Aspirations & Career Plans College Finances Attitudes, Values, & Life Goals Reasons for Attending College  FRESHMAN TRENDS (1966 – present)

9 CIRP FS Trends: Entering College Freshmen

10 CIRP FS Trends: Entering College Freshmen

11 CIRP FS Trends: Entering College Freshmen

12 CIRP FS Trends: Entering College Freshmen Men Women

13 CIRP Freshman Trends Socio-historical context Consistent trends over time Changing student profile over the last 40 years (e.g, values, reasons for going to college, etc.) What about students’ college experiences?

14 CSS HERI follow ups since 1967 Sponsored research College Student Survey Launched in 1993 Allowed institutions to participate on their own timetable Most use CSS as an exit survey for seniors

15 CSS Major Themes Academic and social adjustment Sources of satisfaction and dissatisfaction Academic, residential, and employment experiences Plans for the next academic year Patterns of behavior Life goals Self-concept and feelings of personal success

CSS* *Data are unweighted. 116 Colleges and Universities 17,929 students 85% took the CIRP Freshman Survey in % Female 38% Male

17 Results: 2005 CSS Higher Satisfaction 4 out of 5 would make the same choice over again to attend their college Class size (89.3%) Quality of Instruction (88.0%) Major courses (86.5%) Sense of community (76.9%) Ability to find faculty/staff mentor (67.5%)

18 Results: 2005 CSS Higher Satisfaction Internet access (81.2%) Computer facilities (73.3%) Opportunities for Community Service (59.2%) Leadership opportunities (59.6%)

19 Results: 2005 CSS Lower Satisfaction Student Housing (54.0%) Campus Health Services (43.7%) Job Placement (36.4%)

20 CSS (by racial group): Overall Satisfaction w/College Experience

21 Students felt successful in…

22 CIRP vs. CSS (longitudinal results) How do students change over their years of college?

23 Longitudinal Expectations/Reality: CIRP Freshman Survey and CSS

24 Longitudinal Expectations/Reality: CIRP Freshman Survey and CSS

25 Longitudinal Results: CIRP Freshman Survey & CSS Change in Students' Self-Concept Percent rating themselves "above average" or "highest 10% relative to their peers :At college entryAt the end of collegeChange Computer skills Self-understanding Writing ability Self-confidence (social) Public speaking ability Self-confidence (intellectual) Leadership ability Understanding of others Popularity Artistic ability Creativity Competitiveness Academic ability

26 Longitudinal Results: CIRP Freshman Survey & CSS Change in Students' Self-Concept Percent rating themselves "above average" or "highest 10% relative to their peers :At college entryAt the end of collegeChange Drive to achieve Emotional health Spirituality Physical health Religiousness Mathematical ability

27 Using CIRP – CSS at University of New Hampshire

28 Tradition of Using CIRP Freshman Survey at UNH UNH was pilot school in 1966 Data for 16 years going back to 1966 Have administered biennially since 1999 Have used data to better understand incoming students Haven’t been using data for any type of evaluation or outcomes assessment o UNH is lacking specific outcomes

29 College Student Survey at UNH In 2001 UNH began to look more at student outcomes o Focus on learning, not just satisfaction and general programmatic outcomes such as attendance Viewed CSS as a tool for that assessment

30 College Student Survey at UNH Compared CSS to NSSE and chose CIRP/CSS o High response rate for CIRP o Control of pre-college factors with CIRP o Great deal of question overlap between CIRP/CSS and NSSE o CIRP/CSS was cheaper for UNH o Ability to pre-/post-test with same panel of students, not just sample from same cohorts  Need SSNs o 30 extra questions on CIRP/CSS to customize for our campus

31 College Student Survey at UNH CSS administration challenges o Inadequate mail/ addresses in 2001 o Tried to have departments administer in capstone experiences and other activities where seniors were together  Only three departments participated o Lack of intentional purpose for using data once collected  UNH academic plan is general not specific enough to evaluate against  We are in the process of identifying institutional student outcomes

32 College Student Survey at UNH CSS administration spring 2005 o Pilot test with 789 students who took CIRP in 2001 and that provided SSNs for linkage o Able to reach 763 of 789 students from this sampling frame  97% coverage rate o 278 of 763 students responded  36% response rate

33 Using CIRP/CSS Data at UNH Understanding student characteristics Needs assessment Satisfaction survey of various campus services Outcomes assessment Benchmarking Program evaluation Strategic planning

34 Using CIRP/CSS Data at UNH Understanding student characteristics o How they spent their time, behaviors, values, etc. and how this has changed over their college career  CIRP 2006: 27, 32, 33, 36, 38  CSS 2006: 7, 8, 13, 15, 21 o Educational activities engaged in at college  CSS 2006: 7, 9

35 Using CIRP/CSS Data at UNH Needs assessment o Areas of self-identified weakness o CIRP item 28, 39 Satisfaction survey of various campus services o CSS 2006: 10, 18

36 Using CIRP/CSS Data at UNH Outcomes assessment o Achievement of UNH student outcomes  Can use extra 30 questions  Can investigate relationships between variables to identify possible predictors for outcomes while controlling for pre-college inputs o Examining how students’ perceived skill level/traits has changed during college career  CIRP 2006: 31/CSS 2006: 23  CSS 2006: 19

37 Using CIRP/CSS Data at UNH Outcomes assessment (continued) o Understanding the perceived impact of faculty  CSS 2006: 24

38 Using CIRP/CSS Data at UNH Benchmarking against other institutions on any variety of variables Program evaluation o Can compare students in certain programs based on CSS variables, or change between CIRP and CSS on particular variables  Academic program  Co-curricular program

39 Using CIRP/CSS Data at UNH Strategic planning Helping to assess selected goals and strategies of the strategic plan Increase opportunities for student engagement  CSS 2006: 7, 9, 13, 19 Strengthen and facilitate sense of community  CSS 2006: 10, 18, 22, 24, 26

40 Using CIRP/CSS Data at UNH Strategic planning (continued) Helping to assess selected goals and strategies of the strategic plan Actively develop multicultural competence and support diversity Establish and maintain a healthy/safe environment for students and staff  CIRP 2006: 26/CSS 2006: 15  CSS 2006: 26

41 Fostering use of CIRP/CSS at UNH Commitment to assessment and evidence- based planning and decision-making Concrete goals, strategies, and outcomes that can be evaluated Marketing how the data can be used Putting the data into practice for assessment, planning, and decision-making

42 For More Information: Victor John Gavin (310)