Maximizing Your NSSE & CCSSE Results Jillian Kinzie Associate Director NSSE Nathan Marti Senior Research Associate CCSSE

Agenda Why is assessment of student engagement important? What is student engagement? NSSE & CCSSE fundamentals Interpreting NSSE and CCSSE data Using benchmarks Q & A SDSU Assessment Conference NSSE/CCSSE Workshop

Introductory Activity What questions do you have coming into this session? Assessment at your institution: What do you want to know about your students? Why do you want to know this? What is the purpose of your assessment initiative(s)? To what extent have you used NSSE and CCSSE data? SDSU Assessment Conference NSSE/CCSSE Workshop

Why assessment of student engagement? “Colleges and universities, for all the benefits they bring, accomplish far less for their students than they should.” -- Derek Bok “We can tell people almost anything about education except how well students are learning.” -- Patrick M. Callan, President, National Center for Public Policy and Higher Education SDSU Assessment Conference NSSE/CCSSE Workshop

What Do We Need to Know? What constitutes quality in undergraduate education? How might we measure quality? What types of evidence of quality would be helpful in guiding improvement efforts? SDSU Assessment Conference NSSE/CCSSE Workshop

Foundations of Student Engagement Quality of effort (Pace) Student involvement (Astin) Social and academic integration (Tinto) Good practices in undergraduate education (Chickering & Gamson) Learning and development model (Pascarella) Pace (1970s) Pioneer in this movement of looking at the entire student experience versus just looking at test scores or grades to assess student learning. He asked questions related to students’ academic and social experiences in college – assessed how much effort they were putting forth in their educational experiences Astin (Hired by Pact at UCLA in 1980s) Promoted his theory of student involvement Amount of learning taking place directly proportional to quantity and quality of energy invested in educational activities Tinto (Also in the 1980s) Retention model – greater social and academic integration, both formal and informal processes -> greater satisfaction -> more likely to stay Chickering and Gamson (1980s analysis of hundreds of studies over several decades) 1) Student-faculty contact, 2) Cooperation among students, 3) Active learning, 4) Prompt feedback, 5) Time on task, 6) High expectations, and 7) Respect for diverse talents and ways of learning Kuh (1990s – idea of student engagement) What students do -- time and energy devoted to educationally purposeful activities What institutions do -- using effective educational practices to induce students to do the right things SDSU Assessment Conference NSSE/CCSSE Workshop

Agreement within the Research What matters most is what students do, not who they are. Educationally effective institutions channel student energy toward the right activities. SDSU Assessment Conference NSSE/CCSSE Workshop

Principles of Good Practice in Undergraduate Education Student-faculty contact Active learning Prompt feedback Time on task High expectations Experiences with diversity Cooperation among students (Chickering & Gamson, 1987) SDSU Assessment Conference NSSE/CCSSE Workshop

Path Model for Assessing Change in Student Learning (Pascarella) Pascarella (1985) general causal model for assessing the effects of differential college environments on student learning and cognitive development SDSU Assessment Conference NSSE/CCSSE Workshop

What Really Matters in College: Student Engagement Because individual effort and involvement are the critical determinants of college impact, institutions should focus on the ways they can shape their academic, interpersonal, and extracurricular offerings to encourage student engagement. Pascarella & Terenzini, How College Affects Students, 2005, p. 602 SDSU Assessment Conference NSSE/CCSSE Workshop

Two Assertions The time and energy students devote to educationally purposeful activities is the single best predictor of their learning and personal development. These things can be influenced and measured by institutions. SDSU Assessment Conference NSSE/CCSSE Workshop

Two Components of Student Engagement What students do – time and energy devoted to educationally purposeful activities 2. What institutions do – using effective educational practices to induce students to do the right things SDSU Assessment Conference NSSE/CCSSE Workshop

Educational Process Indicators National Survey of Student Engagement (pronounced “nessie”) Community College Survey of Student Engagement (pronounced “sessie”) College student surveys that assess the extent to which students engage in educational practices associated with high levels of learning and development SDSU Assessment Conference NSSE/CCSSE Workshop

NSSE – The College Student Report Based on effective educational practices Items have face and content validity Designed & tested for validity and reliability Relatively stable over time Credibility of self-reported data Students will participate Actionable data The Center for Survey Research at Indiana University helps NSSE administer the survey utilizing best practices in the field. See: http://www.iub.edu/~csr/ for more information. NSSE design parameters: Relatively short survey Items directly related to college outcomes Administered to first-year and senior students at public and private 4-yr institutions Administered directly by a credible third-party survey organization Validity of Self-Reporting Improves When… Requested information is known to respondents Questions are clear and unambiguous Respondents take questions seriously and thoughtfully Answering does not threaten, embarrass, or violate privacy or compel a socially desirable response SDSU Assessment Conference NSSE/CCSSE Workshop

NSSE Administration Centrally administered by third party Spring term Random samples of first-year & senior students Administration modes: Paper Web-only Web + 40% response rate Over 275,000 students at 600 institutions annually 40+ consortium, state systems SDSU Assessment Conference NSSE/CCSSE Workshop

NSSE’s Institutional Report NSSE Overview Institutional data Respondent characteristics Means and frequency comparisons by selected peers, Carnegie, national Using NSSE results Accreditation Toolkits, Facilitators Guide PowerPoint presentation SDSU Assessment Conference NSSE/CCSSE Workshop

Benchmark Reports Effective Educational Practices Level of Academic Challenge Active and Collaborative Learning Student-Faculty Interaction Enriching Educational Experiences Supportive Campus Environment SDSU Assessment Conference NSSE/CCSSE Workshop

What have we learned from NSSE so far? The single best predictor of student satisfaction with college is the degree to which they perceive the college environment to be supportive of their academic and social needs. Effective educational practices measured by NSSE are independent of institutional selectivity. Schools that have a lower student-faculty ratio, more full-time faculty, and more classes with fewer than 20 students generally score higher on all five NSSE benchmarks. Grades, persistence, student satisfaction, and engagement go hand in hand. SDSU Assessment Conference NSSE/CCSSE Workshop

Grades, persistence, and engagement go hand in hand SDSU Assessment Conference NSSE/CCSSE Workshop

Benchmark Scores for All Students by Undergraduate Enrollment SDSU Assessment Conference NSSE/CCSSE Workshop

Academic Challenge at Two Public Universities Student engagement varies more within than between institutions. SDSU Assessment Conference NSSE/CCSSE Workshop

Relationship Between NSSE & Graduation Rates Benchmarks of Effective Educational Practice First-year students* Seniors* Academic Challenge .60 .46 Active & Collaborative Learning .23 .09 Student Faculty Interaction .28 .37 Enriching Educational Experiences .53 .48 Supportive Campus Environment .38 .26 *All correlations are significant at p<.01 SDSU Assessment Conference NSSE/CCSSE Workshop

NSSE Promising Findings More than 75% of “A” students say they are highly motivated to succeed compared with only half of the “C” students. At institutions where faculty members use effective educational practices more frequently in their classes, students are more engaged over all and gain more from college. A majority of students (54% of first-year students and 63% of seniors) says they often discuss ideas from readings or classes with others outside of class, and well over 90% do this at least sometimes. SDSU Assessment Conference NSSE/CCSSE Workshop

NSSE Results Are diagnostic; to help institutions look holistically at undergraduate experience Help pinpoint aspects not in line with mission, or what institution expects Identify weaknesses and strengths in educational program Help institutions know what to focus on to improve student learning and success SDSU Assessment Conference NSSE/CCSSE Workshop

Questions to answer with NSSE results How many hours per week do first-year students spend studying? Do women study more than men? What % of seniors work with faculty members on activities other than coursework (activities, committees)? Does this differ by major? What % of FY and SR spend 0 hours in co- curricular involvements? Is this more than at peer institutions? Do FY students work more frequently with classmates on assignments outside of class than their counterparts at peer institutions? SDSU Assessment Conference NSSE/CCSSE Workshop

Questions to answer with NSSE results cont’d Do NSSE results match our mission and what we say about a [INSTITUTION] experience? Are we meeting our own expectations for having a supportive campus environment? Since implementing a new multicultural education initiative and expanding diversity programming, has our score on the diversity scale changed? Are FY who withdraw from the institution different in terms of engagement than students who are retained? How are we performing compared to our select peers (normative benchmarking) or to our institutionally identified standards (criterion benchmarking)? SDSU Assessment Conference NSSE/CCSSE Workshop

NSSE suite of instruments Beginning College Survey of Student Engagement National Survey of Student Engagement Faculty Survey of Student Engagement Annual survey of first-year students at baccalaureate degree-granting institutions 2005-2006 pilot administration at 70 institutions Administered prior to start of classes, usually at orientation or welcome week Parallel survey to measure faculty expectations for student engagement 2006 administration at 131 institutions Administered during the spring semester Annual survey of first-year and senior students 2006 administration at 571 institutions Administered during the spring semester Since 2000 NSSE has received add’l funding from sources including the Lumina Foundation for Education and the Wabash College Center of Inquiry in the Liberal Arts SDSU Assessment Conference NSSE/CCSSE Workshop 27

Faculty Survey of Student Engagement (FSSE) Parallels NSSE content Fosters campus-wide discussions of teaching and learning To date more than 53,000 faculty members at over 300 four-year institutions Faculty perceptions of student engagement Importance faculty place on various areas of learning and development Nature and frequency of faculty-student interactions How faculty members organize class time The Center for Survey Research at Indiana University helps NSSE administer the survey utilizing best practices in the field. See: http://www.iub.edu/~csr/ for more information. NSSE design parameters: Relatively short survey Items directly related to college outcomes Administered to first-year and senior students at public and private 4-yr institutions Administered directly by a credible third-party survey organization SDSU Assessment Conference NSSE/CCSSE Workshop

Beginning College Survey of Student Engagement – BCSSE New! Beginning College Survey of Student Engagement – BCSSE Designed for entering first-year students as a companion to NSSE Measures: pre-college academic and co-curricular experiences expectations for educationally purposeful activities during college Amy SDSU Assessment Conference NSSE/CCSSE Workshop 29

BCSSE Instrument Three pilots: ‘04, ‘05, and ‘06 2004 pilot with 28 schools, 15,890 students 2005-2006 pilots with 80 institutions, 39,986 students Study effect of students’ background on NSSE scores Use to examine gap between expectations and engagement Registration now open Amy SDSU Assessment Conference NSSE/CCSSE Workshop 30

NSSE Accreditation Toolkit SDSU Assessment Conference NSSE/CCSSE Workshop

NSSE Accreditation Toolkit SDSU Assessment Conference NSSE/CCSSE Workshop

Connecting NSSE Data to Accreditation Standards - Example Accreditation standard: “Demonstrate effectiveness of student academic and social support services” Evidence for institutional self study: Information about availability and student use of tutoring, writing support, peer study groups, counseling services NSSE indicates FY & SR believe institution emphasizes spending time studying and support for student success; 79% seniors tutored or taught peers; positive correlation between peer collaboration outside of class, satisfaction and first-year retention Positive student satisfaction data about support services Area for improvement - seniors indicate low gains in writing and completing drafts of papers; institution responds with examination of writing requirement in senior capstone and targets seniors for increased use of writing center SDSU Assessment Conference NSSE/CCSSE Workshop

Institutional Example: NSSE and Enrollment Management The enrollment management area at Meredith has used NSSE results to help guide the enrollment marketing strategies. They look closely at trends and make adjustments to programs and campus visitation days to ensure that students are more cognizant of student involvement and engagement opportunities. An academic dean reports using NSSE information when speaking to parents at an admissions event. "Parents seemed impressed that there was data to support the points that I was making about what we say about the student/faculty relationships and educational opportunities at Meredith." SDSU Assessment Conference NSSE/CCSSE Workshop

SDSU Assessment Conference NSSE/CCSSE Workshop

Institutional Example: Hanover College A detailed summary of NSSE is sent to the faculty as well as the Admission and Student Life staffs to ensure the results, both good and bad, are understood by key folks on campus. Last year, Admission requested an additional presentation and discussion of findings to help them better understand the strengths of the Hanover experience and how that impacts student fit. SDSU Assessment Conference NSSE/CCSSE Workshop

Institutional Example: NSSE & Assessing General Education goals Used NSSE items in 11a-p to assess institutional impact on college-level competencies (a.k.a., indirect measures of student learning outcomes) Undergraduate seniors 2005 NSSE results confirmed findings from 2004 Most seniors (75%+) reported that KSU experience had “substantial impact” (VM+QAB) in 9 or 16 college-level competencies KSU rank ordered competencies, showing connection to mission, and compared to other master’s instit where KSU was sig. higher, comparable, sig. lower on competencies SDSU Assessment Conference NSSE/CCSSE Workshop

Institutional Example: Program Development and Strategic Planning NSSE results framed a “Sophomore Experience” 2005 = Pace’s 5th year of participation Concern regarding SP- JR persistence; FY results offers context for understanding exp. as students enter SP year Established “SP Experience Working Group” to investigate if FY exp. carried over in SP year. Focused on low NSSE score items, conducted focus groups, created sophomore survey. Led to pilot of “Pace Plan” (mentoring), includes Career Exploration Course, Sophomore Kick-Off Day NSSE also used in strategic indicators, Accred, NCATE, AACSB, Faculty Development/Colloquia, items used by offices (Technology, Multicultural Affairs), studies performed by Enrollment Mngmt. SDSU Assessment Conference NSSE/CCSSE Workshop

CCSSE History In 2001, received 3-year grant funding from Lumina Foundation and Pew Charitable Trusts to design, pilot and administer a community college survey of student engagement Pilot administration in the fall of 2001 - 12 institutions participated National Field Test in the spring of 2002 - 48 institutions participated Annual, national, administrations in the spring of each year: 2003 - 93 institutions 2004 - 152 institutions 2005 - 257 institutions 2006-193 institutions 2007-279 institutions Self-funding project beginning September 2004 SDSU Assessment Conference NSSE/CCSSE Workshop

CCSSE’s Survey Administration Stratified random sample of credit courses Stratification – morning, afternoon, evening In-class, paper and pencil administration Average administration time = 35 minutes Completed surveys returned to CCSSE for scanning and analyses SDSU Assessment Conference NSSE/CCSSE Workshop

Community College Faculty Survey of Student Engagement (CCFSSE) Pilot test in fall of 2004 First national administration in spring of 2005 Over 20,000 participants SDSU Assessment Conference NSSE/CCSSE Workshop

CCSSE Institutional Report CCSSE Overview College Results: A Synopsis Means Summary Frequency Distributions Benchmark Reports Using CCSSE Results CCFSSE Codebook Appendix SDSU Assessment Conference NSSE/CCSSE Workshop

CCSSE Annual Report Review of CCSSE Benchmarks Active and Collaborative Learning Student Effort Academic Challenge Student-Faculty Interaction Support for Learners Focus on trends and issues for community and technical colleges SDSU Assessment Conference NSSE/CCSSE Workshop

Overview of 2005 National Survey Results More than 133,000 community college students from 257 community and technical colleges in 38 states responded to the 2005 CCSSE survey. Since 2002, more than 320,000 students in 404 colleges across 43 states have responded to CCSSE. Consortia for 2005 include statewide participation in Indiana, Louisiana, North Dakota, Virginia, and West Virginia. Other state-based consortia include groups of colleges in Georgia, Illinois, Minnesota, North Carolina, and Northeast Minnesota. SDSU Assessment Conference NSSE/CCSSE Workshop

Institutional Use: Example Estrella Mountain (AZ) Received CCSSE results in the summer of 2004 Implemented an intensive improvement program that included student engagement in the college-wide assessment process Student engagement is now an Estrella Mountain Core Indicator of Effectiveness for Student Success Conducted three improvement sessions with students, faculty and staff; discussed the implications of CCSSE at the annual strategic planning retreat, and identified five improvement strategies for increasing student engagement. Developing a freshmen institute (with learning communities), Improving publicity of workshops and services available to students (through the use of electronic kiosks), Developing student life activities to increase contact with faculty in non-academic settings, Creating facilities to improve adjunct faculty interaction with students, and; Integrating student support services into the course curriculum René G. Willekens, Dean of Planning, Research and Effectiveness SDSU Assessment Conference NSSE/CCSSE Workshop

Institutional Use: Example Northwest Vista College (TX) CCSSE results are used in the Quality Enhancement Plan (QEP) at Northwest Vista College (NVC). NVC’s QEP concentrates on assessing student learning outcomes for the core curriculum by serving as a focused area of NVC’s Attitude, Skills, and Knowledge (ASK) model. CCSSE supports both NVC’s QEP and ASK by providing information about NVC’s student learning outcomes, including critical thinking, cooperative learning, and student collaboration; CCSSE survey items directly and indirectly relate to many of the 12 student learning outcomes identified as important by NVC faculty. Christa Emig, Math Instructor SDSU Assessment Conference NSSE/CCSSE Workshop

Institutional Use: Example North Hennepin Community College (MN) We designed an exercise on the web site in which faculty are asked to guess how students answered CCSSE questions. After guessing, they see the real answer. Instead of doing this on paper, we used personal response system "clickers" and got immediate feedback after guessing at each question. The data could be displayed in several ways for discussion. We had staff, faculty, and administration very engaged! We used questions from each of the benchmark groupings, and after getting everyone interested in the data, we divided them randomly into benchmark groups and asked them to identify 2 priorities for change at the college in that area. These are now worked into our assessment plan initiatives. Mary Diedrich, Active Learning and Assessment Coordinator SDSU Assessment Conference NSSE/CCSSE Workshop

Questions?