1. Maximizing Your NSSE & CCSSE Results
Jillian Kinzie
Associate Director
NSSE
Nathan Marti
Senior Research Associate
CCSSE

2. Agenda Why is assessment of student engagement important?
What is student engagement?
NSSE & CCSSE fundamentals
Interpreting NSSE and CCSSE data
Using benchmarks
Q & A
SDSU Assessment Conference NSSE/CCSSE Workshop

3. Introductory Activity
What questions do you have coming into this session?
Assessment at your institution:
What do you want to know about your students?
Why do you want to know this?
What is the purpose of your assessment initiative(s)?
To what extent have you used NSSE and CCSSE data?
SDSU Assessment Conference NSSE/CCSSE Workshop

4. Why assessment of student engagement?
“Colleges and universities, for all the benefits they bring, accomplish far less for their students than they should.” Derek Bok
“We can tell people almost anything about education except how well students are learning.” -- Patrick M. Callan, President, National Center for Public Policy and Higher Education
SDSU Assessment Conference NSSE/CCSSE Workshop

5. What Do We Need to Know?
What constitutes quality in undergraduate education?
How might we measure quality?
What types of evidence of quality would be helpful in guiding improvement efforts?
SDSU Assessment Conference NSSE/CCSSE Workshop

6. Foundations of Student Engagement
Quality of effort (Pace)
Student involvement (Astin)
Social and academic integration (Tinto)
Good practices in undergraduate education (Chickering & Gamson)
Learning and development model (Pascarella)
Pace (1970s)
Pioneer in this movement of looking at the entire student experience versus just looking at test scores or grades to assess student learning.
He asked questions related to students’ academic and social experiences in college – assessed how much effort they were putting forth in their educational experiences
Astin (Hired by Pact at UCLA in 1980s)
Promoted his theory of student involvement
Amount of learning taking place directly proportional to quantity and quality of energy invested in educational activities
Tinto (Also in the 1980s)
Retention model – greater social and academic integration, both formal and informal processes -> greater satisfaction -> more likely to stay
Chickering and Gamson (1980s analysis of hundreds of studies over several decades)
1) Student-faculty contact, 2) Cooperation among students, 3) Active learning, 4) Prompt feedback, 5) Time on task, 6) High expectations, and 7) Respect for diverse talents and ways of learning
Kuh (1990s – idea of student engagement)
What students do -- time and energy devoted to educationally purposeful activities
What institutions do -- using effective educational practices to induce students to do the right things
SDSU Assessment Conference NSSE/CCSSE Workshop

7. Agreement within the Research
What matters most is what students do, not who they are.
Educationally effective institutions channel student energy toward the right activities.
SDSU Assessment Conference NSSE/CCSSE Workshop

8. Principles of Good Practice in Undergraduate Education
Student-faculty contact
Active learning
Prompt feedback
Time on task
High expectations
Experiences with diversity
Cooperation among students
(Chickering & Gamson, 1987)
SDSU Assessment Conference NSSE/CCSSE Workshop

9. Path Model for Assessing Change in Student Learning (Pascarella)
Pascarella (1985) general causal model for assessing the effects of differential college environments on student learning and cognitive development
SDSU Assessment Conference NSSE/CCSSE Workshop

10. What Really Matters in College: Student Engagement
Because individual effort and involvement are the critical determinants of college impact, institutions should focus on the ways they can shape their academic, interpersonal, and extracurricular offerings to encourage student engagement.
Pascarella & Terenzini, How College Affects Students, 2005, p. 602
SDSU Assessment Conference NSSE/CCSSE Workshop

11. Two Assertions
The time and energy students devote to educationally purposeful activities is the single best predictor of their learning and personal development.
These things can be influenced and measured by institutions.
SDSU Assessment Conference NSSE/CCSSE Workshop

12. Two Components of Student Engagement
What students do – time and energy devoted to educationally purposeful activities
2. What institutions do – using effective educational practices to induce students to do the right things
SDSU Assessment Conference NSSE/CCSSE Workshop

13. Educational Process Indicators
National Survey of Student Engagement (pronounced “nessie”) Community College Survey of Student Engagement (pronounced “sessie”)
College student surveys that assess the extent to which students engage in educational practices associated with high levels of learning and development
SDSU Assessment Conference NSSE/CCSSE Workshop

14. NSSE – The College Student Report
Based on effective educational practices
Items have face and content validity
Designed & tested for validity and reliability
Relatively stable over time
Credibility of self-reported data
Students will participate
Actionable data
The Center for Survey Research at Indiana University helps NSSE administer the survey utilizing best practices in the field. See: for more information.
NSSE design parameters:
Relatively short survey
Items directly related to college outcomes
Administered to first-year and senior students at public and private 4-yr institutions
Administered directly by a credible third-party survey organization
Validity of Self-Reporting Improves When…
Requested information is known to respondents
Questions are clear and unambiguous
Respondents take questions seriously and thoughtfully
Answering does not threaten, embarrass, or violate privacy or compel a socially desirable response
SDSU Assessment Conference NSSE/CCSSE Workshop

15. NSSE Administration Centrally administered by third party Spring term
Random samples of first-year & senior students
Administration modes:
Paper
Web-only
Web +
40% response rate
Over 275,000 students at institutions annually
40+ consortium, state systems
SDSU Assessment Conference NSSE/CCSSE Workshop

16. NSSE’s Institutional Report
NSSE Overview
Institutional data
Respondent characteristics
Means and frequency comparisons by selected peers, Carnegie, national
Using NSSE results
Accreditation Toolkits, Facilitators Guide
PowerPoint presentation
SDSU Assessment Conference NSSE/CCSSE Workshop

17. Benchmark Reports Effective Educational Practices
Level of Academic Challenge
Active and Collaborative Learning
Student-Faculty Interaction
Enriching Educational Experiences
Supportive Campus Environment
SDSU Assessment Conference NSSE/CCSSE Workshop

18. What have we learned from NSSE so far?
The single best predictor of student satisfaction with college is the degree to which they perceive the college environment to be supportive of their academic and social needs.
Effective educational practices measured by NSSE are independent of institutional selectivity.
Schools that have a lower student-faculty ratio, more full-time faculty, and more classes with fewer than 20 students generally score higher on all five NSSE benchmarks.
Grades, persistence, student satisfaction, and engagement go hand in hand.
SDSU Assessment Conference NSSE/CCSSE Workshop

19. Grades, persistence, and engagement go hand in hand
SDSU Assessment Conference NSSE/CCSSE Workshop

20. Benchmark Scores for All Students by Undergraduate Enrollment
SDSU Assessment Conference NSSE/CCSSE Workshop

21. Academic Challenge at Two Public Universities
Student engagement varies more within than between institutions.
SDSU Assessment Conference NSSE/CCSSE Workshop

22. Relationship Between NSSE & Graduation Rates
Benchmarks of Effective Educational Practice
First-year students*
Seniors*
Academic Challenge
.60
.46
Active & Collaborative Learning
.23
.09
Student Faculty Interaction
.28
.37
Enriching Educational Experiences
.53
.48
Supportive Campus Environment
.38
.26
*All correlations are significant at p<.01
SDSU Assessment Conference NSSE/CCSSE Workshop

23. NSSE Promising Findings
More than 75% of “A” students say they are highly motivated to succeed compared with only half of the “C” students.
At institutions where faculty members use effective educational practices more frequently in their classes, students are more engaged over all and gain more from college.
A majority of students (54% of first-year students and 63% of seniors) says they often discuss ideas from readings or classes with others outside of class, and well over 90% do this at least sometimes.
SDSU Assessment Conference NSSE/CCSSE Workshop

24. NSSE Results
Are diagnostic; to help institutions look holistically at undergraduate experience
Help pinpoint aspects not in line with mission, or what institution expects
Identify weaknesses and strengths in educational program
Help institutions know what to focus on to improve student learning and success
SDSU Assessment Conference NSSE/CCSSE Workshop

25. Questions to answer with NSSE results
How many hours per week do first-year students spend studying? Do women study more than men?
What % of seniors work with faculty members on activities other than coursework (activities, committees)? Does this differ by major?
What % of FY and SR spend 0 hours in co- curricular involvements? Is this more than at peer institutions?
Do FY students work more frequently with classmates on assignments outside of class than their counterparts at peer institutions?
SDSU Assessment Conference NSSE/CCSSE Workshop

26. Questions to answer with NSSE results cont’d
Do NSSE results match our mission and what we say about a [INSTITUTION] experience?
Are we meeting our own expectations for having a supportive campus environment?
Since implementing a new multicultural education initiative and expanding diversity programming, has our score on the diversity scale changed?
Are FY who withdraw from the institution different in terms of engagement than students who are retained?
How are we performing compared to our select peers (normative benchmarking) or to our institutionally identified standards (criterion benchmarking)?
SDSU Assessment Conference NSSE/CCSSE Workshop

27. NSSE suite of instruments
Beginning College Survey of Student Engagement
National Survey of Student Engagement
Faculty Survey of Student Engagement
Annual survey of first-year students at baccalaureate degree-granting institutions
pilot administration at 70 institutions
Administered prior to start of classes, usually at orientation or welcome week
Parallel survey to measure faculty expectations for student engagement
2006 administration at 131 institutions
Administered during the spring semester
Annual survey of first-year and senior students
2006 administration at 571 institutions
Administered during the spring semester
Since 2000 NSSE has received add’l funding from sources including the Lumina Foundation for Education and the Wabash College Center of Inquiry in the Liberal Arts
SDSU Assessment Conference NSSE/CCSSE Workshop 27

28. Faculty Survey of Student Engagement (FSSE)
Parallels NSSE content
Fosters campus-wide discussions of teaching and learning
To date more than 53,000 faculty members at over 300 four-year institutions
Faculty perceptions of student engagement
Importance faculty place on various areas of learning and development
Nature and frequency of faculty-student interactions
How faculty members organize class time
The Center for Survey Research at Indiana University helps NSSE administer the survey utilizing best practices in the field. See: for more information.
NSSE design parameters:
Relatively short survey
Items directly related to college outcomes
Administered to first-year and senior students at public and private 4-yr institutions
Administered directly by a credible third-party survey organization
SDSU Assessment Conference NSSE/CCSSE Workshop

29. Beginning College Survey of Student Engagement – BCSSE
New!
Beginning College Survey of Student Engagement – BCSSE
Designed for entering first-year students as a companion to NSSE
Measures:
pre-college academic and co-curricular experiences
expectations for educationally purposeful activities during college
Amy
SDSU Assessment Conference NSSE/CCSSE Workshop 29

30. BCSSE Instrument Three pilots: ‘04, ‘05, and ‘06
2004 pilot with 28 schools, 15,890 students
pilots with 80 institutions, 39,986 students
Study effect of students’ background on NSSE scores
Use to examine gap between expectations and engagement
Registration now open
Amy
SDSU Assessment Conference NSSE/CCSSE Workshop 30

31. NSSE Accreditation Toolkit
SDSU Assessment Conference NSSE/CCSSE Workshop

32. NSSE Accreditation Toolkit
SDSU Assessment Conference NSSE/CCSSE Workshop

33. Connecting NSSE Data to Accreditation Standards - Example
Accreditation standard: “Demonstrate effectiveness of student academic and social support services”
Evidence for institutional self study:
Information about availability and student use of tutoring, writing support, peer study groups, counseling services
NSSE indicates FY & SR believe institution emphasizes spending time studying and support for student success; 79% seniors tutored or taught peers; positive correlation between peer collaboration outside of class, satisfaction and first-year retention
Positive student satisfaction data about support services
Area for improvement - seniors indicate low gains in writing and completing drafts of papers; institution responds with examination of writing requirement in senior capstone and targets seniors for increased use of writing center
SDSU Assessment Conference NSSE/CCSSE Workshop

34. Institutional Example: NSSE and Enrollment Management
The enrollment management area at Meredith has used NSSE results to help guide the enrollment marketing strategies. They look closely at trends and make adjustments to programs and campus visitation days to ensure that students are more cognizant of student involvement and engagement opportunities.
An academic dean reports using NSSE information when speaking to parents at an admissions event. "Parents seemed impressed that there was data to support the points that I was making about what we say about the student/faculty relationships and educational opportunities at Meredith."
SDSU Assessment Conference NSSE/CCSSE Workshop

35. SDSU Assessment Conference NSSE/CCSSE Workshop

36. Institutional Example: Hanover College
A detailed summary of NSSE is sent to the faculty as well as the Admission and Student Life staffs to ensure the results, both good and bad, are understood by key folks on campus. Last year, Admission requested an additional presentation and discussion of findings to help them better understand the strengths of the Hanover experience and how that impacts student fit.
SDSU Assessment Conference NSSE/CCSSE Workshop

37. Institutional Example: NSSE & Assessing General Education goals
Used NSSE items in 11a-p to assess institutional impact on college-level competencies (a.k.a., indirect measures of student learning outcomes)
Undergraduate seniors 2005 NSSE results confirmed findings from 2004
Most seniors (75%+) reported that KSU experience had “substantial impact” (VM+QAB) in 9 or 16 college-level competencies
KSU rank ordered competencies, showing connection to mission, and compared to other master’s instit where KSU was sig. higher, comparable, sig. lower on competencies
SDSU Assessment Conference NSSE/CCSSE Workshop

38. Institutional Example: Program Development and Strategic Planning
NSSE results framed a “Sophomore Experience”
2005 = Pace’s 5th year of participation
Concern regarding SP- JR persistence; FY results offers context for understanding exp. as students enter SP year
Established “SP Experience Working Group” to investigate if FY exp. carried over in SP year. Focused on low NSSE score items, conducted focus groups, created sophomore survey. Led to pilot of “Pace Plan” (mentoring), includes Career Exploration Course, Sophomore Kick-Off Day
NSSE also used in strategic indicators, Accred, NCATE, AACSB, Faculty Development/Colloquia, items used by offices (Technology, Multicultural Affairs), studies performed by Enrollment Mngmt.
SDSU Assessment Conference NSSE/CCSSE Workshop

39. CCSSE History
In 2001, received 3-year grant funding from Lumina Foundation and Pew Charitable Trusts to design, pilot and administer a community college survey of student engagement
Pilot administration in the fall of institutions participated
National Field Test in the spring of institutions participated
Annual, national, administrations in the spring of each year:
institutions
institutions
institutions
institutions
institutions
Self-funding project beginning September 2004
SDSU Assessment Conference NSSE/CCSSE Workshop

40. CCSSE’s Survey Administration
Stratified random sample of credit courses
Stratification – morning, afternoon, evening
In-class, paper and pencil administration
Average administration time = 35 minutes
Completed surveys returned to CCSSE for scanning and analyses
SDSU Assessment Conference NSSE/CCSSE Workshop

41. Community College Faculty Survey of Student Engagement (CCFSSE)
Pilot test in fall of 2004
First national administration in spring of 2005
Over 20,000 participants
SDSU Assessment Conference NSSE/CCSSE Workshop

42. CCSSE Institutional Report
CCSSE Overview
College Results: A Synopsis
Means Summary
Frequency Distributions
Benchmark Reports
Using CCSSE Results
CCFSSE
Codebook
Appendix
SDSU Assessment Conference NSSE/CCSSE Workshop

43. CCSSE Annual Report Review of CCSSE Benchmarks
Active and Collaborative Learning
Student Effort
Academic Challenge
Student-Faculty Interaction
Support for Learners
Focus on trends and issues for community and technical colleges
SDSU Assessment Conference NSSE/CCSSE Workshop

44. Overview of 2005 National Survey Results
More than 133,000 community college students from 257 community and technical colleges in 38 states responded to the 2005 CCSSE survey.
Since 2002, more than 320,000 students in 404 colleges across 43 states have responded to CCSSE.
Consortia for 2005 include statewide participation in Indiana, Louisiana, North Dakota, Virginia, and West Virginia. Other state-based consortia include groups of colleges in Georgia, Illinois, Minnesota, North Carolina, and Northeast Minnesota.
SDSU Assessment Conference NSSE/CCSSE Workshop

45. Institutional Use: Example
Estrella Mountain (AZ)
Received CCSSE results in the summer of 2004
Implemented an intensive improvement program that included student engagement in the college-wide assessment process
Student engagement is now an Estrella Mountain Core Indicator of Effectiveness for Student Success
Conducted three improvement sessions with students, faculty and staff; discussed the implications of CCSSE at the annual strategic planning retreat, and identified five improvement strategies for increasing student engagement.
Developing a freshmen institute (with learning communities),
Improving publicity of workshops and services available to students (through the use of electronic kiosks),
Developing student life activities to increase contact with faculty in non-academic settings,
Creating facilities to improve adjunct faculty interaction with students, and;
Integrating student support services into the course curriculum
René G. Willekens, Dean of Planning, Research and Effectiveness
SDSU Assessment Conference NSSE/CCSSE Workshop

46. Institutional Use: Example
Northwest Vista College (TX)
CCSSE results are used in the Quality Enhancement Plan (QEP) at Northwest Vista College (NVC).
NVC’s QEP concentrates on assessing student learning outcomes for the core curriculum by serving as a focused area of NVC’s Attitude, Skills, and Knowledge (ASK) model.
CCSSE supports both NVC’s QEP and ASK by providing information about NVC’s student learning outcomes, including critical thinking, cooperative learning, and student collaboration;
CCSSE survey items directly and indirectly relate to many of the 12 student learning outcomes identified as important by NVC faculty.
Christa Emig, Math Instructor
SDSU Assessment Conference NSSE/CCSSE Workshop

47. Institutional Use: Example
North Hennepin Community College (MN)
We designed an exercise on the web site in which faculty are asked to guess how students answered CCSSE questions. After guessing, they see the real answer. Instead of doing this on paper, we used personal response system "clickers" and got immediate feedback after guessing at each question. The data could be displayed in several ways for discussion. We had staff, faculty, and administration very engaged!
We used questions from each of the benchmark groupings, and after getting everyone interested in the data, we divided them randomly into benchmark groups and asked them to identify 2 priorities for change at the college in that area. These are now worked into our assessment plan initiatives.
Mary Diedrich, Active Learning and Assessment Coordinator
SDSU Assessment Conference NSSE/CCSSE Workshop

48. Questions?