nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement Alexander C. McCormick Center for Postsecondary Research Indiana University Bloomington, USA HEPI-HEA Conference The British Academy, London 22 May 2012

Context: Conceptions of quality in U.S. tertiary education The National Survey of Student Engagement (NSSE) Low stakes, high yield assessment Student engagement trends Controversies and cautions Overview

nsse.iub.eduNational Survey of Student Engagement Context Higher education in the USA Limited understanding of quality Student engagement as a new approach

nsse.iub.eduNational Survey of Student Engagement Higher Education in the USA  No national system of higher education  Autonomy of institutions  Diversity of institutions  4,600 degree-granting tertiary institutions  Public and private  Universities (undergraduate and postgraduate)  Colleges (undergraduate only) 4-year bachelor’s degree 2-year associate’s degree (community colleges)

nsse.iub.eduNational Survey of Student Engagement Higher Education in the USA (2) Diversity of students  Full- and part-time  Traditional college-age  Older and returning students  Widely varying levels of preparation

nsse.iub.eduNational Survey of Student Engagement What Do We Really Know About University Quality?

nsse.iub.eduNational Survey of Student Engagement Quality Assessment in US Higher Education  Accreditation  Limited public understanding  Limited disclosure  Traditional emphasis on capacity and resources  Limited attention to teaching & learning (until recently)  Newsmagazine rankings  Emphasize reputation, resources, other inputs  Research rankings  Blind to undergraduate education

nsse.iub.eduNational Survey of Student Engagement Origins of the “National” Survey of Student Engagement (NSSE)  A reaction to the dominant discourse about university quality in the USA  Shift the focus to teaching and learning  Promote evidence- based improvement

nsse.iub.eduNational Survey of Student Engagement The Big Idea  Ask students about their experience  Focus on behavior: empirically confirmed effective practices

nsse.iub.eduNational Survey of Student Engagement Why a Student Survey? Direct assessment of learning is problematic  Measurement challenges  Inferential challenges  Cost challenges  Motivation challenges Survey approach offers advantages  Cost-effective and efficient  Diagnostic utility of behavioral data  Students are the experts on their experience

nsse.iub.eduNational Survey of Student Engagement NSSE’s Key Aims Enrich the impoverished national discourse about university quality  Shift the focus to teaching & learning, through the lens of effective practices Provide diagnostic information that can be used to improve undergraduate education  Actionable information based on valid & reliable measures  Meaningful comparisons to other institutions

nsse.iub.eduNational Survey of Student Engagement What is Student Engagement? The extent to which students engage in and are exposed to effective educational practices  Activities and practices known to be related to desired outcomes

nsse.iub.eduNational Survey of Student Engagement OK, but what is Engagement?  Challenging academic work  Active learning  Deep approaches to learning  High-impact practices  Involvement with instructors In a campus context that promotes and supports success

nsse.iub.eduNational Survey of Student Engagement Conceptual & Empirical Foundations Time on task (Tyler, 1930s) Quality of effort (Pace, 1960s & 70s) Student involvement (Astin, 1984) Academic & social integration (Tinto, 1975 & 1987) Good practices in undergraduate education (Chickering & Gamson, 1987) College impact (Pascarella, 1985) Student engagement (Kuh, 1991, 2005)

nsse.iub.eduNational Survey of Student Engagement Why Does Engagement Matter? The impact of college is largely determined by individual effort and involvement in the academic, interpersonal, and extra-curricular offerings on a campus. It is important to focus on the ways an institution can shape its offerings to encourage student engagement. Paraphrased from Pascarella & Terenzini (2005), p. 602

How Does it Work? What Do We Ask?

nsse.iub.eduNational Survey of Student Engagement How NSSE Works  Paper or Web-based survey  First- & final-year undergraduates  Uniform, centralized survey administration  Institutions receive detailed reports and an identified student data file  Customizable comparison groups  Results are confidential  Supported by institution fees

nsse.iub.eduNational Survey of Student Engagement Participation  550 to 770 U.S. & Canadian institutions per year  About 1,500 since NSSE’s launch in 2000  Most repeat every 1-4 years

nsse.iub.eduNational Survey of Student Engagement Survey Content  Academic activities & experiences Academic challenge Reading and writing Active learning Cognitive tasks in courses Enriching educational experiences  Time use  Study, work, socializing…  Co-curricular activities  Institutional emphases  Quality of campus relationships  Cognitive and noncognitive gains  Satisfaction  Demographic & enrollment characteristics

nsse.iub.eduNational Survey of Student Engagement Sample Questions  Asked questions in class or contributed to class discussions?  Made a class presentation?  Prepared two or more drafts of a paper or assignment before turning it in? In your experience at your institution during the current school year, about how often have you…

nsse.iub.eduNational Survey of Student Engagement Sample Questions  Put together ideas or concepts from different courses when completing assignments or during class discussions?  Discussed ideas from your readings or classes with faculty members outside of class? In your experience at your institution during the current school year, about how often have you…

nsse.iub.eduNational Survey of Student Engagement Sample Questions  Received prompt written or oral feedback from faculty on your academic performance?  Worked harder than you thought you could to meet an instructor’s standards or expectations? In your experience at your institution during the current school year, about how often have you…

nsse.iub.eduNational Survey of Student Engagement Sample Questions  Memorising facts, ideas, or methods…  Analysing the basic elements of an idea…  Synthesising and organizing ideas, information…  Making judgments about the value of information…  Applying theories or concepts… During the current school year, how much has your coursework emphasized the following mental activities? During the current school year, how much has your coursework emphasized the following mental activities?

nsse.iub.eduNational Survey of Student Engagement Sample Questions  Examined the strengths or weaknesses of your own views on a topic or issue?  Learned something that changed the way you understand an issue or concept? During the current school year, about how often have you done each of the following?

nsse.iub.eduNational Survey of Student Engagement Using NSSE Results How are we doing? How can we improve?

nsse.iub.eduNational Survey of Student Engagement Using NSSE Results  Diagnostic purpose: analysing the undergraduate experience  Identify strengths and weaknesses, areas where improvement is needed  Stimulates conversations about quality  Informs decision-making with “actionable” information

nsse.iub.eduNational Survey of Student Engagement Many Ways to Use Results Peer comparisons  How do we measure up to others? Self comparison  Are we improving? Absolute standard  How do we measure up to our ideals? Internal variation  Who is least engaged? What to look at and how to look at it

NSSE results as a catalyst for campus discussions about teaching & learning

nsse.iub.eduNational Survey of Student Engagement Comparing Student and Instructor Perceptions Courses emphasised memorisation “Quite a bit” or “Very much” Faculty Survey

nsse.iub.eduNational Survey of Student Engagement Comparing Student and Instructor Perceptions Courses emphasised memorisation “Quite a bit” or “Very much”

Results at institutions with several NSSE administrations show many with positive trends

nsse.iub.eduNational Survey of Student Engagement Summary of Findings 41% of institutions demonstrated a pattern of improvement in at least one measure for first- year students 28% did so for final-year students Trends found across many institutional types  Control (public & private), Size, Research emphasis Ratio of positive to negative trends: 7:1

nsse.iub.eduNational Survey of Student Engagement Overview of Observed Changes: FY

nsse.iub.eduNational Survey of Student Engagement Overview of Observed Changes: Sr

nsse.iub.eduNational Survey of Student Engagement Cautions & Controversies NSSE is no “magic bullet” The risks of high-stakes uses “Participation” vs. “use”

nsse.iub.eduNational Survey of Student Engagement Cautions  NSSE is no “magic bullet”: Best used in combination with other information  NSSE’s research foundation is based on studies of US university students  Most variation is between students, not institutions

nsse.iub.eduNational Survey of Student Engagement Controversies  Confidentiality of results: Institutions control disclosure  Attaching high stakes to NSSE is risky  Institutions will decline to participate if public relations cost > institutional benefit  Reliability of responses could be compromised  Mere participation is not sufficient  Results must be analysed and used

nsse.iub.eduNational Survey of Student Engagement Understanding Improvement Organizations that improve… create and nurture agreement on what is worth achieving, and they set in motion the internal processes by which people progressively learn how to do what they need to do in order to achieve what is worthwhile. Elmore in Fullan, 2001) (Elmore in Fullan, 2001)

Thank you.