Mining NSS data for enhancement An example from Art & Design Mantz Yorke Higher Education Academy Conference “Higher Education.

Slides:

Advertisements

Similar presentations

Library Review – Heath Park and All Wales Review (Phase 2) Prof. Simon Smail, CBE Emeritus Professor, Chair of Review Committee.

Advertisements

Matching applicants to the best courses - UCAS Entry Profiles and feedback to students Geoff Ramshaw Policy Executive Policy and Communications Department.

Institutional Liaison Scheme Emerging Outcomes: Scotland and the rest of the UK Alastair Robertson Senior Adviser, Scotland Academy Scotland- priorities.

Conceptualising and Testing Generic Skills in an International Assessment Of Higher Education Learning Outcomes Doug McCurry Australian Council for Educational.

Department Underachievement KEYNOTE 6 th Form Conference June 2011.

Gallaudet University Results on National Survey of Student Engagement Office of Institutional Research August, 2007.

Dissertation Writing.

Analysis of Scottish NSS Results 29 th April 2010 Dr Alex Buckley The Higher Education Academy.

Universal Access: More People. More Situations Content or Graphics Content or Graphics? An Empirical Analysis of Criteria for Award-Winning Websites Rashmi.

School of something FACULTY OF OTHER Faculty of Arts ‘Fair, prompt & detailed’ – matching staff and student expectations on assessment and feedback in.

Improving Students’ understanding of Feedback

Copyright ©2015 Pearson Education, Inc Strategy Review, Evaluation, and Control Chapter Nine 9-1.

Recognition as a Professional Teacher in Higher Education: The Higher Education Academy professional recognition scheme Helen Barefoot Deputy Head of the.

What is HEA Fellowship? What’s the UK PSF?

‘Belongingness’, engagement and self-confidence in first-year students Mantz Yorke Lancaster University This work is supported.

Quality teaching in higher education, an OECD approach Fabrice Hénard, December 2010.

Slide 1 of 17 Lessons from the Foundation Learning provision for the new 16 to 19 Study Programmes Discussion materials Issue 2: The development of English.

Understanding the postgraduate experience Chris Park Director, Lancaster University Graduate School Senior Associate, Higher Education Academy (HEA)

School Innovation in Science Formerly Science in Schools An overview of the SIS Model & supporting research Russell Tytler Faculty of Education, Deakin.

NSS: Components of Institutional Experience 29 th April 2010 Dr Alex Buckley The Higher Education Academy.

Northampton – Development Opportunities a framework for enabling positive change.

Thinking Actively in a Social Context T A S C.

Overall Teacher Judgements

LSE 2009 Staff Survey – Presentation to Staff Briefings 15 th /16 th March 2010.

Selection Process Information Workshop Teaching Staff.

Slide 1 of 19 Lessons from the Foundation Learning provision for the new 16 to 19 Study Programmes Discussion materials Issue 1: Attendance, retention,

University Strategy Marcus Williams. Contents 0.1 Strategic goals 0.2 Re-positioning 0.3 Campus infrastructure 0.4 Sussex off campus 0.5 Malaysia Office.

Improving Practical Work in Science Session 1 Secondary.

1 What Students Need to Know from The National Student Survey 17 June 2010 Sami Benyahia, Director.

Using student evaluations to improve individual and department teaching qualities Mary Hedges (Auckland University, NZ) Don Webber (University of the West.

Enhancing student learning through assessment: a school-wide approach Christine O'Leary, Centre for Promoting Learner Autonomy Sheffield Business School.

Centre for Educational Development ORHEP Project 1 This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported.

The contrasting environments that early career academics experience in their departmental teaching and on programmes of initial professional development.

How Colleges Improve – (from ‘LSIS Supporting Improvement Conference’ 19 September 2012) Session outline Survey background Successful and improving’ ‘Decline.

Quality Assurance and Enhancement at Middlesex University Ian Pearson, Head of Academic Quality.

Key features of the University of Manchester Professor Cathy Cassell Deputy Director (Academic) Sarah Featherstone Head of Undergraduate Services Original.

ToK ESSAY The instructions tell you to: Remember to centre your essay on knowledge issues and,where appropriate, refer to other parts of your IB programme.

GLAD NSS RESEARCH PROJECT ‘I can’t believe it’s not better’: The Paradox of NSS scores for Art & Design David Vaughan and Mantz Yorke This research was.

Postgraduate Taught Experience Survey (PTES) 2010 Interim Results Dr Pam Wells Adviser, Evidence-Informed Practice.

Jason Leman Education Researcher Sheffield Hallam University.

National Student Survey MMU Overview Neil Barrett Strategic Planning & Management Information.

Directorate of Human Resources The use of the Nominal Group Technique in evaluating student experience Diana Williams OCSLD.

© New Zealand Ministry of Education copying restricted to use by New Zealand education sector. Page 1 Consider the Evidence Evidence-driven.

© 2001 Change Function Ltd USER ACCEPTANCE TESTING Is user acceptance testing of technology and / or processes a task within the project? If ‘Yes’: Will.

Clare Saunders and Danielle Lamb Subject Centre for Philosophical and Religious Studies.

Extracting useful information from the UK’s National Student (Satisfaction) Survey Mark Langan, Alan Fielding and Peter Dunleavy Manchester Metropolitan.

National Student Survey Outcomes Medicine 2014 Professor Lindsay Bashford Director of Academic Undergraduate Studies.

Developing competition briefs CPD2d How to use this resource: To develop and enhance skills in the design and assessment of skills competition briefs for.

LSD Inspection Development Project Workshop on self-assessment using CIF and new inspection handbook Sue Preece HMI NATSPEC Conference September 14 th.

An Assessment For Learning. A rubric is a scoring tool that lists the criteria for a piece of work, or “what counts” and clearly defines gradations of.

Towards a Citizens’ #citizenscurriculum Alex Stevenson, NIACE Addison Barnett, St Mungo’s Broadway Dermot.

to become a critical consumer of information.

National PE Cycle of Analysis. Fitness Assessment + Gathering Data Why do we need to asses our fitness levels?? * Strengths + Weeknesses -> Develop Performance.

Middle School Survey 2010 Selection of evidence from the Survey.

Using the NSS to enhance teaching quality 22 nd June 2011 Dr Alex Buckley The Higher Education Academy.

Assessment Instruments and Rubrics Workshop Series Part 1: What is a rubric? What are the required elements? What are VALUE rubrics? February 24, 2016.

LCF Green Academy The GA programme formed a network of HEIs, and offered support from the HEA and external specialists to enable teams to embed sustainability.

Annual report findings 2010/11 Lorna Fitzjohn, Divisional Manager, Learning and Skills 29 November 2011.

Supported by: Student Hubs is a registered charity in England and Wales, number Impact Measurement Proving and Improving your Social Impact.

Developing a revised approach to considering subject level NSS data – a Durham example Richard Harrison Head of the Academic Support Office 15 April 2011.

Quality Enhancement Report Learning Initiative

Postgraduate Taught Experience Survey (PTES) 2010 Interim Results

What is HEA Fellowship? What’s the UK PSF?

Assessment and Feedback – Module 1

SCHOOL OF ART DESIGN & MEDIA NSS POSITIVITY 2016

Pre-service and in-service teachers experiences of learning to program in an object-oriented language Presenter: Ku-Chou Tai Advisor: Ming-Puu Chen Govender,

Graduating Excellent Clinicians

Consider the Evidence Evidence-driven decision making

Marzenna Hiles Why are Creative and Media undergraduates in the UK reporting lower satisfaction rates with their courses?

Evidence for Enhancement: Making sense of surveys

Presentation transcript:

Mining NSS data for enhancement An example from Art & Design Mantz Yorke Higher Education Academy Conference “Higher Education Surveys for Enhancement” Nottingham, 18 May 2010

This presentation illustrates one way in which NSS data have been used in the interests of enhancement It ought to be adaptable to other situations

Art & Design makes a substantial contribution to the national economy. It is a national success story. Yet when it is viewed through the lens of the NSS, it does not come out well in comparison with other subject areas (save in respect of Assessment & Feedback). Question 1: Why? The paradox of Art & Design

Judged by NSS scores, some institutions do much better than others. Question 2: Why? Variation in the provision of Art & Design

I am critical of the NSS, on various conceptual and technical grounds. It is problematic as a basis for informing the public. It is however more useful in its ancillary role as a prompt to enhancing the student experience. Even if it’s a gross ‘measure’, we can still get messages from it. Institutions are ill-advised to ‘chase the ratings’: rather, they need to look underneath the scores to identify points at which enhancement-oriented activity might most profitably be targeted. Clearing some (substantial) brushwood

The NSS is part of a political agenda: it is here to stay (for the time being, at least – but the focus of interest is beginning to switch to student engagement, following the use of NSSE in the US and AUSSE in Australia). So we have to live with it, even if we don’t like it. Politics

Art & Design and the NSS

Teaching Asst & Fdback Acad Sup Org & Man Learn Res Pers Dev OvS Selected subject comparisons, NSS 2009 (New universities and colleges in England and Wales) Timetable Changes Organisation Explaining Interesting Enthusiastic Stimulating Prompt Detailed Clarified Specialist equipment NB Means are unweighted

Teaching Asst & Fdback Acad Sup Org & Man Learn Res Pers Dev OvS Selected subject comparisons, NSS 2009 (New universities and colleges in England and Wales) Timetable Changes Organisation Explaining Interesting Enthusiastic Stimulating Prompt Detailed Clarified Specialist equipment NB Means are unweighted

GLAD, the HE Academy ADM Subject Centre and the HEAD Trust combined in supporting a study of the reasons underlying the relatively poor performance of Art & Design since the inception of the NSS. The study had quantitative and qualitative aspects: only the former is represented in this presentation. (See the HEA-ADM Subject Centre website for the report of the study.) The NSS and Art & Design – a case study

Disaggregating to institutions with studio-based provision Design Studies is used as an illustration: similar results are found for Fine Art and Cine & Photo Studies

Specialist A&D institution Some institutions do (consistently) better than others No ‘specialist institution effect’ No particular size effect

Digging deeper into the NSS data Three examples

Prompt Detailed Clarified Timetable Changes Organisation Explaining Interesting Enthusiastic Stimulating Specialist equipment Design Studies, selected comparisons (NSS 2009) Teaching Asst & Fdback Acad Sup Org & Man Learn Res Pers Dev OvSNovel problems Clear criteria

Fine Art: selected comparisons (NSS 2009) Teaching Asst & Fdback Acad Sup Org & Man Learn Res Pers Dev OvS Prompt Detailed Clarified Timetable Changes Organisation Explaining Interesting Enthusiastic Stimulating Specialist equipment Novel problems

English: selected comparisons (NSS 2009) Teaching Asst & Fdback Acad Sup Org & Man Learn Res Pers Dev OvS Prompt Detailed Clarified Timetable Changes Organisation Explaining Interesting Enthusiastic Stimulating Specialist equipment Novel problems

Some possible determinants of the NSS outcomes Specific institutional and/or departmental causes (e.g. relocation, staffing difficulties). Programme leadership and/or management. Pedagogic approach in Art & Design. Curricular positioning of Art & Design in (some) multi-disciplinary institutions. Balance of FT and PT staff. Student expectations v. experiences: ‘The Deal’. A&D students having to be more self-organising than many others as regards booking spaces and resources; self-organising may cut across formal timetabling.

Some methodological points The illustrations are based on raw data. ‘Residuals’ make allowances for various institutional factors, such as entry profile, but their interpretation and presentation to others require statistical sophistication. The raw data may be rougher than one would like, because they are affected by local considerations and the limits of the NSS as an instrument. One year’s NSS scores may therefore be unreliable. Nevertheless, rough and ready data may be ‘good enough’ to act as prompts to enhancement. Warning Shaping the data into a usable form is a bit of a slog.

In conclusion NSS data show relative strengths and weaknesses. The ‘quality literature’ asserts that there’s almost always something that can be improved, by doing things better or doing things differently. The important thing is to identify those aspects of provision where enhancement activity is most likely to pay off, and to focus on them. There are unlikely to be many quick fixes.