1. Measuring Student Success: The Value and Limitations of NSSE Chris Conway, Queen’s University HEQCO/MTCU/OISE Symposium on Defining and Measuring Student Success November 22, 2013

2. Overview NSSE – The National Survey of Student Engagement – is a survey of undergraduates that measures a range of student behaviours and institutional practices that research has shown to be associated with knowledge acquisition, skills development and personal growth (i.e. learning) 2

3. It is a key component of success/outcomes/quality discussions and practices – MYA’s, strategic plan monitoring, academic program reviews, course-based evaluations (CLASSE), accreditations, external communications, the media,... But how well does it really fare as a student success metric? – Strengths – Weaknesses – Complexity of interpretation and application 3

4. Strengths Empirically and viscerally connects the dots between process and outcomes Implementation options and priorities are reasonably clear, particularly with comparative data It’s consistent with accepted approaches to accountability It’s here-and-now 4

5. Weaknesses Results are not 1:1 prescriptive and therefore cannot predict specific LO effects or even identify specific LO dimensions The most useful form of results (program- and student subgroup-level against single comparators), is cumbersome and requires a translator/intermediary Theoretically stronger but politically weaker than certain hot button metrics (e.g. labour market outcomes) in measuring learning rather than the assumed impacts of learning 5

6. Complexities Program- and student subgroup-specific engagement dynamics Actual vs. predicted engagement Benchmark vs. component item behaviour Individual institution vs. peer group comparisons Few general statements about NSSE results are true 6

7. The Context for Student Success Measures Last week’s HEQCO blog by Tricia Seifert and others suggests a lack of consensus on what we mean by student success and how we should measure it There certainly are lots of options, and NSSE seems to fill a useful role within an overall outcomes metrics framework 7

8. 8

9. Final Thoughts – There is no measurement panacea: student success requires multiple metrics because it is multi-dimensional; “certainty” emerges when all the metrics point in the same direction – Even direct outcomes measures (e.g. CLA) suffer limitations (particularly complexity) – NSSE is a guide – along with other tools - to navigating part of the LO landscape, recognizing that engagement doesn’t fully explain outcomes 9

10. Measuring Student Success: The Value and Limitations of NSSE Chris Conway, Queen’s University HEQCO/MTCU/OISE Symposium on Defining and Measuring Student Success November 22, 2013