Learning from the NSS: can’t get no satisfaction…… "This year's National Student Survey is a wake-up call to university vice-chancellors. They must buck up their ideas and do far more to improve the experience they offer students.” Aaron Porter NUS president HEA NSS Institutional Working Group Dr Clare Milsom Head of Learning and Teaching Academic Enhancement Unit Liverpool John Moores University

Very quantitative approach Sector: Fielding et al. (2010); Lutz and Culver (2010); Yorke (2009) Marsh and Cheng (2008); Richardson (2007). Questionnaire robustness: construct validity; internal consistency; concurrent validity ‘understand the subtleties in the information provided by the NSS and how it can be used to inform decision making’ (Fielding et al. 2010, 350) ‘The complexity of student satisfaction...reliable measures...enhance the student experience’ (Fielding et al. 2010, 349) ‘where a survey is intended to be used formatively, such as the identification of areas of the student experience that might benefit from enhancement activities, the extent of reflection on the instrument’s technical qualities can arguably be relatively limited’ Yorke (2009, 735)

Female > 25Male < 25 Part timeFull time PostgraduateUndergraduate First yearLater years Williams and Kane (2008) ‘Further qualitative work should be undertaken to gain a better understanding’ NSS Assessment and Feedback More satisfiedLess satisfied > 25others InternationalEU Qualification other than first degree First degree HEFEC 2010 Findings and Trends: Student, course and institutional characteristics Subject% diff Education81 ITT79-2 Social Studies810 Physical Science887 Creative Arts & Design 72-9 Biological Sciences854 Geographical Studies886

Teaching on my course % Assess and feedback % Academic support % Organisation and management % Learning resources % Personal development % Overall Satisfaction % 2010 Diff Diff Diff Diff Diff Diff Diff 09 80%-460%-471%-678%-280%-775%-682%-5 75%62%070%-474%-272%74%-379%-2 80%-365%-771%-550%-1473%-585%277%2 70%252%169%174%678%-373%476%6 79%065%-371%73%2 -373%-274%-3 76%-559%-467%663%-367%-778%-472%-6 Faculty Very quantitative approach Sector: Fielding et al. (2010); Lutz and Culver (2010); Yorke (2009) Richardson (2007); Faculty response proforma 2010 introduced requirement to discuss qualitative data

Qualitative analysis of NSS Student comments Stage 1:12 themes identified Stage 2:Frequency analysis - strengths / weaknesses - comment alignment Stage 3:Meta analysis With Martyn Stewart and Elena Zaitseva Academic Enhancement, LJMU

Subject Reporting Group Positive Frequency Positive Strength Negative Frequency Negative Strength Teaching Assessment Feedback Academic support Timetabling and deadlines Course organisation + facilities LRC + access to PCs PDP + skills Employability + placement ICT Accessibility issues VFM

ThemeSummary TeachingPositive enthusiasm / expertise Negative enthusiasm / skills AssessmentDeadline bunching FeedbackDelays Academic SupportStrong positive relationships - marking bias. Course organisationNegative- disruption communication and repetition in cross school programmes Value for moneyIssue for programmes with low contact hours Meta analysis

Comment alignment: subject reporting group Content analysis: frequency Discourse analysis: strength – needs more work

Priorities for Academic Enhancement ‘Focus on the what steps need to be taken to move forward and improve rather than concentrating on over analysing the scores and focusing solely on the past situation’ Flint et al. (2009, 615) ‘ ‘Aspirational...participatory... evidence-based QE approaches.’ Operate a ‘surplus’ model What would make you change practice? How does dissemination of ‘best practice’ work best?