Dr Alex Buckley 16 February 2017, University of Stirling Satisfaction versus engagement: Using student feedback to improve programmes Dr Alex Buckley 16 February 2017, University of Stirling

Plan for talk Three different types of programme-level student surveys Formative and summative uses of programme-level student feedback Ideas for the formative use of programme-level student feedback

Three different types of programme-level student surveys Student perceptions of quality Student satisfaction Student engagement

A. Student perceptions of quality “…based on a theory of university teaching and learning in which students' perceptions of curriculum, instruction and assessment are regarded as key determinants of their approaches to learning and the quality of their learning outcomes” (Wilson et al 1997)

B. Student satisfaction “[T]he emphasis of the Student Satisfaction approach is on the ‘customer’ rather than ‘producer’ or ‘supplier’ view. That is, it is the point of view of students as participants in a process of education that the Student Satisfaction approach is intending to assess rather than the student satisfaction with those aspects of education that the producers (teachers or administrators) think are of concern” (Harvey 1995)

C. Student engagement Student involvement (Astin 1984) Integration (Tinto 1975) Deep vs surface approaches to learning (Marton and Saljo 1976) Quality of effort (Pace 1984) Seven principles of good practice (Chickering and Gamson 1987) “College is basically a voluntary activity… This is why the quality of effort, which one might also think of as quality of initiative, is so important at the college level… [C]ollege can’t give you an education; but if you go to college, and fully use the facilities and opportunities it provides, you can get an education.” (Pace 1982)

Satisfaction vs engagement “The term [‘student engagement’] covers two domains relating to: improving the motivation of students to engage in learning and to learn independently the participation of students in quality enhancement and quality assurance processes, resulting in the improvement of their educational experience” (QAA 2012)

Satisfaction vs engagement “The NSS put the student voice firmly on the agenda of every higher education institution, and has prompted radical changes in how students are taught and treated in institutions up and down the country.” (NUS 2015)

2. Formative and summative uses of programme-level student feedback

3. Ideas for the formative use of programme-level student surveys Good feedback practice: Facilitates the development of self-assessment (reflection) in learning Encourages positive motivational beliefs and self-esteem Provides opportunities to close the gap between current and desired performance (Nicola and Macfarlane-Dick 2006)

3. Ideas for the formative use of programme-level student feedback “The crucial variable appears not to be the quality of the feedback (which is what teachers tend to focus on) but the quality of student engagement with that feedback” (Gibbs 2006) “[T]here is an assumption that when teachers transmit feedback information to students these messages are easily decoded and translated into action. Yet, there is strong evidence that feedback messages are invariably complex and difficult to decipher, and that students require opportunities to construct actively an understanding of them (e.g. through discussion) before they can be used to regulate performance” (Nicol and Macfarlane-Dick 2006)

3. Ideas for the formative use of programme-level student feedback “The crucial variable appears not to be the quality of the feedback (which is what teachers tend to focus on) but the quality of student engagement with that feedback” (Gibbs 2006) “[T]here is an assumption that when students transmit feedback information to staff these messages are easily decoded and translated into action. Yet, there is strong evidence that feedback messages are invariably complex and difficult to decipher, and that staff require opportunities to construct actively an understanding of them (e.g. through discussion) before they can be used to regulate performance” (Nicol and Macfarlane-Dick 2006)

3. Ideas for the formative use of programme-level student feedback “Acting on student feedback does not necessarily mean doing what students say they want. The teacher should remain in control. This does not imply, however, any dismissal of students’ comments. It means understanding their meaning in relation to the whole course” (Ramsden 2003)

3. Ideas for the formative use of programme-level student feedback “Evaluation is best conceptualised not as something that is done to teachers by experts wielding questionnaires and spreadsheets, but as something that is done by teachers for the benefit of their professional competence and their students’ understanding” (Ramsden 2003)

References Astin, A. (1984) “Student involvement: A developmental theory for higher education” Journal of College Student Development 25(4): 297-308 Chickering, A. and Gamson, Z. (1987) “Seven principles for good practice in undergraduate education” AAHE Bulletin March: 3-7 Ewell, P. (2009) Assessment, accountability, and improvement: Revisiting the tension. Champaign IL, NILOA Gibbs, G. (2006) ‘How assessment frames student learning’ in Bryan, C. and Klegg, K. (eds.) Innovative assessment in higher education (Abingdon, Routledge) Harvey, L. (1995) Keeping the customer satisfied: The student satisfaction approach (University of Central England, QHE) Jessop, T., Tomas, C. (2016) ‘The implications of programme assessment patterns for student learning’ Assessment and Evaluation in Higher Education Marton, F. and Saljo, R. (1976) “On qualitative differences in learning – 1: Outcome and process” British Journal of Educational Psychology 46: 4-11 Nicol, D. and Macfarlane-Dick, D. (2006) ‘Formative assessment and self-regulated learning: a model and seven principles of good feedback practice’ Studies in Higher Education 31(2): 199-218 NUS (2013) The future of the National Student Survey. London, National Union of Students Pace, R. (1982) Achievement and the quality of student effort (Department of Education, Washington DC) Pace, C. (1984) Measuring the quality of college student experiences (University of California, Los Angeles) QAA. (2012). UK Quality Code for Higher Education, Chapter B5: Student engagement. Gloucester, Quality Assurance Agency. Ramsden, P. (2003) Learning to teach in higher education (Abingdon, Routledge Falmer) Tinto, V. (1975). “Dropouts from higher education: A theoretical synthesis of the recent literature”. Review of Educational Research, 45 , 89–125. Wilson, K., Lizzio, A. and Ramsden, P. (1997) “The development, validation and application of the Course Experience Questionnaire” Studies in Higher Education 22(1): 33-53 . . . . . . . . . .