Presentation on theme: "Maximizing Your NSSE & CCSSE Results Jillian Kinzie Associate Director NSSE Nathan Marti Senior Research Associate CCSSE."— Presentation transcript:
Maximizing Your NSSE & CCSSE Results Jillian Kinzie Associate Director NSSE Nathan Marti Senior Research Associate CCSSE
SDSU Assessment Conference NSSE/CCSSE Workshop Agenda Why is assessment of student engagement important? What is student engagement? NSSE & CCSSE fundamentals Interpreting NSSE and CCSSE data Using benchmarks Q & A
SDSU Assessment Conference NSSE/CCSSE Workshop Introductory Activity What questions do you have coming into this session? Assessment at your institution: What do you want to know about your students? Why do you want to know this? What is the purpose of your assessment initiative(s)? To what extent have you used NSSE and CCSSE data?
SDSU Assessment Conference NSSE/CCSSE Workshop “Colleges and universities, for all the benefits they bring, accomplish far less for their students than they should.” -- Derek Bok “We can tell people almost anything about education except how well students are learning.” -- Patrick M. Callan, President, National Center for Public Policy and Higher Education Why assessment of student engagement?
SDSU Assessment Conference NSSE/CCSSE Workshop What Do We Need to Know? What constitutes quality in undergraduate education? How might we measure quality? What types of evidence of quality would be helpful in guiding improvement efforts?
SDSU Assessment Conference NSSE/CCSSE Workshop Foundations of Student Engagement Quality of effort (Pace) Student involvement (Astin) Social and academic integration (Tinto) Good practices in undergraduate education (Chickering & Gamson) Learning and development model (Pascarella)
SDSU Assessment Conference NSSE/CCSSE Workshop Agreement within the Research What matters most is what students do, not who they are. Educationally effective institutions channel student energy toward the right activities.
SDSU Assessment Conference NSSE/CCSSE Workshop Principles of Good Practice in Undergraduate Education Student-faculty contact Active learning Prompt feedback Time on task High expectations Experiences with diversity Cooperation among students (Chickering & Gamson, 1987)
SDSU Assessment Conference NSSE/CCSSE Workshop Path Model for Assessing Change in Student Learning (Pascarella)
SDSU Assessment Conference NSSE/CCSSE Workshop What Really Matters in College: Student Engagement Because individual effort and involvement are the critical determinants of college impact, institutions should focus on the ways they can shape their academic, interpersonal, and extracurricular offerings to encourage student engagement. Pascarella & Terenzini, How College Affects Students, 2005, p. 602
SDSU Assessment Conference NSSE/CCSSE Workshop Two Assertions The time and energy students devote to educationally purposeful activities is the single best predictor of their learning and personal development. These things can be influenced and measured by institutions.
SDSU Assessment Conference NSSE/CCSSE Workshop Two Components of Student Engagement 1.What students do – time and energy devoted to educationally purposeful activities 2. What institutions do – using effective educational practices to induce students to do the right things
SDSU Assessment Conference NSSE/CCSSE Workshop National Survey of Student Engagement (pronounced “nessie”) Community College Survey of Student Engagement (pronounced “sessie”) College student surveys that assess the extent to which students engage in educational practices associated with high levels of learning and development Educational Process Indicators
SDSU Assessment Conference NSSE/CCSSE Workshop NSSE – The College Student Report Based on effective educational practices Items have face and content validity Designed & tested for validity and reliability Relatively stable over time Credibility of self-reported data Students will participate Actionable data
SDSU Assessment Conference NSSE/CCSSE Workshop NSSE Administration Centrally administered by third party Spring term Random samples of first-year & senior students Administration modes: Paper Web-only Web + 40% response rate Over 275,000 students at 600 institutions annually 40+ consortium, state systems
SDSU Assessment Conference NSSE/CCSSE Workshop NSSE’s Institutional Report NSSE Overview Institutional data Respondent characteristics Means and frequency comparisons by selected peers, Carnegie, national Using NSSE results Accreditation Toolkits, Facilitators Guide PowerPoint presentation
SDSU Assessment Conference NSSE/CCSSE Workshop Benchmark Reports Effective Educational Practices Level of Academic Challenge Active and Collaborative Learning Student-Faculty Interaction Enriching Educational Experiences Supportive Campus Environment
SDSU Assessment Conference NSSE/CCSSE Workshop What have we learned from NSSE so far? The single best predictor of student satisfaction with college is the degree to which they perceive the college environment to be supportive of their academic and social needs. Effective educational practices measured by NSSE are independent of institutional selectivity. Schools that have a lower student-faculty ratio, more full-time faculty, and more classes with fewer than 20 students generally score higher on all five NSSE benchmarks. Grades, persistence, student satisfaction, and engagement go hand in hand.
SDSU Assessment Conference NSSE/CCSSE Workshop Grades, persistence, and engagement go hand in hand
SDSU Assessment Conference NSSE/CCSSE Workshop Benchmark Scores for All Students by Undergraduate Enrollment
SDSU Assessment Conference NSSE/CCSSE Workshop Academic Challenge at Two Public Universities Student engagement varies more within than between institutions.
SDSU Assessment Conference NSSE/CCSSE Workshop Benchmarks of Effective Educational Practice First-year students* Seniors* Academic Challenge.60.46 Active & Collaborative Learning.23.09 Student Faculty Interaction.28.37 Enriching Educational Experiences.53.48 Supportive Campus Environment.38.26 Relationship Between NSSE & Graduation Rates *All correlations are significant at p<.01
SDSU Assessment Conference NSSE/CCSSE Workshop NSSE Promising Findings More than 75% of “A” students say they are highly motivated to succeed compared with only half of the “C” students. At institutions where faculty members use effective educational practices more frequently in their classes, students are more engaged over all and gain more from college. A majority of students (54% of first-year students and 63% of seniors) says they often discuss ideas from readings or classes with others outside of class, and well over 90% do this at least sometimes.
SDSU Assessment Conference NSSE/CCSSE Workshop NSSE Results Are diagnostic; to help institutions look holistically at undergraduate experience Help pinpoint aspects not in line with mission, or what institution expects Identify weaknesses and strengths in educational program Help institutions know what to focus on to improve student learning and success
SDSU Assessment Conference NSSE/CCSSE Workshop Questions to answer with NSSE results How many hours per week do first-year students spend studying? Do women study more than men? What % of seniors work with faculty members on activities other than coursework (activities, committees)? Does this differ by major? What % of FY and SR spend 0 hours in co- curricular involvements? Is this more than at peer institutions? Do FY students work more frequently with classmates on assignments outside of class than their counterparts at peer institutions?
SDSU Assessment Conference NSSE/CCSSE Workshop Questions to answer with NSSE results cont’d Do NSSE results match our mission and what we say about a [INSTITUTION] experience? Are we meeting our own expectations for having a supportive campus environment? Since implementing a new multicultural education initiative and expanding diversity programming, has our score on the diversity scale changed? Are FY who withdraw from the institution different in terms of engagement than students who are retained? How are we performing compared to our select peers (normative benchmarking) or to our institutionally identified standards (criterion benchmarking)?
SDSU Assessment Conference NSSE/CCSSE Workshop NSSE suite of instruments Beginning College Survey of Student Engagement National Survey of Student Engagement Annual survey of first-year students at baccalaureate degree-granting institutions 2005-2006 pilot administration at 70 institutions Administered prior to start of classes, usually at orientation or welcome week Annual survey of first-year and senior students 2006 administration at 571 institutions Administered during the spring semester Faculty Survey of Student Engagement Parallel survey to measure faculty expectations for student engagement 2006 administration at 131 institutions Administered during the spring semester
SDSU Assessment Conference NSSE/CCSSE Workshop Faculty Survey of Student Engagement (FSSE) Parallels NSSE content Fosters campus-wide discussions of teaching and learning To date more than 53,000 faculty members at over 300 four-year institutions
SDSU Assessment Conference NSSE/CCSSE Workshop Beginning College Survey of Student Engagement – BCSSE H Designed for entering first-year students as a companion to NSSE H Measures: pre-college academic and co-curricular experiences expectations for educationally purposeful activities during college New!
SDSU Assessment Conference NSSE/CCSSE Workshop BCSSE Instrument Three pilots: ‘04, ‘05, and ‘06 H 2004 pilot with 28 schools, 15,890 students H 2005-2006 pilots with 80 institutions, 39,986 students Study effect of students’ background on NSSE scores Use to examine gap between expectations and engagement Registration now open
SDSU Assessment Conference NSSE/CCSSE Workshop Connecting NSSE Data to Accreditation Standards - Example Accreditation standard: “Demonstrate effectiveness of student academic and social support services” Evidence for institutional self study: Information about availability and student use of tutoring, writing support, peer study groups, counseling services NSSE indicates FY & SR believe institution emphasizes spending time studying and support for student success; 79% seniors tutored or taught peers; positive correlation between peer collaboration outside of class, satisfaction and first-year retention Positive student satisfaction data about support services Area for improvement - seniors indicate low gains in writing and completing drafts of papers; institution responds with examination of writing requirement in senior capstone and targets seniors for increased use of writing center
SDSU Assessment Conference NSSE/CCSSE Workshop Institutional Example: NSSE and Enrollment Management The enrollment management area at Meredith has used NSSE results to help guide the enrollment marketing strategies. They look closely at trends and make adjustments to programs and campus visitation days to ensure that students are more cognizant of student involvement and engagement opportunities. An academic dean reports using NSSE information when speaking to parents at an admissions event. "Parents seemed impressed that there was data to support the points that I was making about what we say about the student/faculty relationships and educational opportunities at Meredith."
SDSU Assessment Conference NSSE/CCSSE Workshop
Institutional Example: Hanover College A detailed summary of NSSE is sent to the faculty as well as the Admission and Student Life staffs to ensure the results, both good and bad, are understood by key folks on campus. Last year, Admission requested an additional presentation and discussion of findings to help them better understand the strengths of the Hanover experience and how that impacts student fit.
SDSU Assessment Conference NSSE/CCSSE Workshop Institutional Example: NSSE & Assessing General Education goals Used NSSE items in 11a-p to assess institutional impact on college-level competencies (a.k.a., indirect measures of student learning outcomes) Undergraduate seniors 2005 NSSE results confirmed findings from 2004 Most seniors (75%+) reported that KSU experience had “substantial impact” (VM+QAB) in 9 or 16 college-level competencies KSU rank ordered competencies, showing connection to mission, and compared to other master’s instit where KSU was sig. higher, comparable, sig. lower on competencies
SDSU Assessment Conference NSSE/CCSSE Workshop Institutional Example: Program Development and Strategic Planning NSSE results framed a “Sophomore Experience” 2005 = Pace’s 5 th year of participation Concern regarding SP- JR persistence; FY results offers context for understanding exp. as students enter SP year Established “SP Experience Working Group” to investigate if FY exp. carried over in SP year. Focused on low NSSE score items, conducted focus groups, created sophomore survey. Led to pilot of “Pace Plan” (mentoring), includes Career Exploration Course, Sophomore Kick-Off Day NSSE also used in strategic indicators, Accred, NCATE, AACSB, Faculty Development/Colloquia, items used by offices (Technology, Multicultural Affairs), studies performed by Enrollment Mngmt.
SDSU Assessment Conference NSSE/CCSSE Workshop CCSSE History In 2001, received 3-year grant funding from Lumina Foundation and Pew Charitable Trusts to design, pilot and administer a community college survey of student engagement Pilot administration in the fall of 2001 - 12 institutions participated National Field Test in the spring of 2002 - 48 institutions participated Annual, national, administrations in the spring of each year: 2003 - 93 institutions 2004 - 152 institutions 2005 - 257 institutions 2006-193 institutions 2007-279 institutions Self-funding project beginning September 2004
SDSU Assessment Conference NSSE/CCSSE Workshop CCSSE’s Survey Administration Stratified random sample of credit courses Stratification – morning, afternoon, evening In-class, paper and pencil administration Average administration time = 35 minutes Completed surveys returned to CCSSE for scanning and analyses
SDSU Assessment Conference NSSE/CCSSE Workshop Community College Faculty Survey of Student Engagement (CCFSSE) Pilot test in fall of 2004 First national administration in spring of 2005 Over 20,000 participants
SDSU Assessment Conference NSSE/CCSSE Workshop CCSSE Institutional Report CCSSE Overview College Results: A Synopsis Means Summary Frequency Distributions Benchmark Reports Using CCSSE Results CCFSSE Codebook Appendix
SDSU Assessment Conference NSSE/CCSSE Workshop CCSSE Annual Report Review of CCSSE Benchmarks Active and Collaborative Learning Student Effort Academic Challenge Student-Faculty Interaction Support for Learners Focus on trends and issues for community and technical colleges
SDSU Assessment Conference NSSE/CCSSE Workshop Overview of 2005 National Survey Results More than 133,000 community college students from 257 community and technical colleges in 38 states responded to the 2005 CCSSE survey. Since 2002, more than 320,000 students in 404 colleges across 43 states have responded to CCSSE. Consortia for 2005 include statewide participation in Indiana, Louisiana, North Dakota, Virginia, and West Virginia. Other state-based consortia include groups of colleges in Georgia, Illinois, Minnesota, North Carolina, and Northeast Minnesota.
SDSU Assessment Conference NSSE/CCSSE Workshop Institutional Use: Example Estrella Mountain (AZ) Received CCSSE results in the summer of 2004 Implemented an intensive improvement program that included student engagement in the college-wide assessment process Student engagement is now an Estrella Mountain Core Indicator of Effectiveness for Student Success Conducted three improvement sessions with students, faculty and staff; discussed the implications of CCSSE at the annual strategic planning retreat, and identified five improvement strategies for increasing student engagement. Developing a freshmen institute (with learning communities), Improving publicity of workshops and services available to students (through the use of electronic kiosks), Developing student life activities to increase contact with faculty in non-academic settings, Creating facilities to improve adjunct faculty interaction with students, and; Integrating student support services into the course curriculum René G. Willekens, Dean of Planning, Research and Effectiveness
SDSU Assessment Conference NSSE/CCSSE Workshop Institutional Use: Example Northwest Vista College (TX) CCSSE results are used in the Quality Enhancement Plan (QEP) at Northwest Vista College (NVC). NVC’s QEP concentrates on assessing student learning outcomes for the core curriculum by serving as a focused area of NVC’s Attitude, Skills, and Knowledge (ASK) model. CCSSE supports both NVC’s QEP and ASK by providing information about NVC’s student learning outcomes, including critical thinking, cooperative learning, and student collaboration; CCSSE survey items directly and indirectly relate to many of the 12 student learning outcomes identified as important by NVC faculty. Christa Emig, Math Instructor
SDSU Assessment Conference NSSE/CCSSE Workshop Institutional Use: Example North Hennepin Community College (MN) We designed an exercise on the web site in which faculty are asked to guess how students answered CCSSE questions. After guessing, they see the real answer. Instead of doing this on paper, we used personal response system "clickers" and got immediate feedback after guessing at each question. The data could be displayed in several ways for discussion. We had staff, faculty, and administration very engaged! We used questions from each of the benchmark groupings, and after getting everyone interested in the data, we divided them randomly into benchmark groups and asked them to identify 2 priorities for change at the college in that area. These are now worked into our assessment plan initiatives. Mary Diedrich, Active Learning and Assessment Coordinator