Use of audio feedback for summative purposes Kathryn McFarlane

Aims Explore a possible rationale for using audio feedback Identify advantages and disadvantages of this approach Explore the practicalities of producing audio feedback Consider potential application strategies in own environment

Outline Rationale for using audio feedback Evaluative project and outcomes The techy bit Application to practice

Rationale for using audio feedback

Activity 2 June 2014 In groups of four, write a list of possible feedback methods on post its Now organise these on the Race grid

Impact on student learning Cost / time High Low High Low

The feedback dilemma.... Feedback (and feed forward) identified as important to learning (e.g. Taras, 2002) Student dissatisfaction with assessment feedback (HEFCE, 2009) Many students do not collect feedback Written feedback – lowest impact on student learning and not cost effective? (Race, 2008)

Why audio feedback? Narrow bandwidth of text based communication - reduces the potential to make judgements about meanings of words (Goodyear, 2001) Greater richness of detail, guidance for academic and professional enhancement, and messages to engage students in thinking (Merry and Orsmond, 2008)

Why audio feedback? Students suggest: Clearer Includes strategies to solve problems, not just problems More personalised Engaged more deeply with it – e.g. using it to prepare for future assignments Tone of voice helpful for highlighting importance of various aspects Accessibility, e.g. can listen while engaging in other activities (Various, e.g. Ribchester et al, 2008; France and Wheeler, 2007; Rotherham, 2008)

Why audio feedback? Inclusion Contemporary students may be less comfortable in processing written information (Merry and Orsmond, 2008) Could be inclusive to Aural learners (Fleming and Baume) Could be inclusive to diverse students, e.g. international, dyslexic (Rotherham, 2008) Potential for inclusive feedback – offering a choice (Waterfield and West, 2008)

Why summative feedback? Disparity between staff and students perceptions: students do use the guidance, and view the mark and supporting feedback as interdependent. (OBrien and Sparshatt 2007) Students initially focus on grade, but use feedback to improve, and some revisited previous work when preparing subsequent assignments (Carless 2006, p.225)

Evaluative project and outcomes

Project background Postgraduate Certificate in Higher and Professional Education – for teaching / learning support staff in HE First module (for most): Supporting Learning – includes a focus on the factors which impact on student learning, and student diversity Participants are mature work- based learners (mainly digital immigrants?)

Evaluation questions What is the impact of using the medium of audio on the messages conveyed by summative feedback? To what extent does audio feedback contribute to feed forward? What is the impact of preparing audio files on tutor workload?

Evaluation Action Research approach – collaborating with the participants Methods: Expert walk through Initial questionnaire pre-feedback On line summative questionnaire Focus groups PebblePad e-journal More details soon to be published in Innovative Practice in HE Journal

Formative evaluation results Guidance on project organisation from expert walkthrough, e.g. reducing file sizes, limiting feedback time Guidance on accessibility from participants (e.g. using PebblePad rather than Blackboard) Audio more explanatory, provided more examples, in everyday language, linked to learning from module

Summative evaluation results…. Problems with verbal delivery – e.g. muffled Grade point at the end – would have preferred it at the beginning Grade point at the end – would have preferred it at the beginning Access and technical problems Not having any text Lack of time to engage with the feedback

...the thing is, with audio feedback, I started playing it, and I thought Well I want to know what mark Ive got, so I didnt listen to about % of it, I just went straight to the end...to see what grade Id got. And then I went back to the beginning again and then listened to it then. (Participant A, lines 13-17) B

if it was a bit of paper, you could have scanned it couldnt you, got the result (Participant B, lines 208-9) B

Summative evaluation results…. Novel format for receiving feedback More detailed comments Strategies gained for future assignments - feed forward Strategies gained for future assignments - feed forward Impact on professional practice Personalisation Explaining academic language Improvements needed were clear Voice giving added meaning

I think what Ill do then, when the next one comes up, is Ill listen to that audio feedback again, Ill have my assignment in front of me, and Ill actually make notes on the assignment, and Ill refer to that, and use the format for the next assignment. If its appropriate, Ill do that. And then Ill feel more confident then. (Participant A, lines ) B

......it almost felt like he had taken the time out to actually assess that piece of work, whereas usually when its paper feedback, its almost like a factory produced thing. (Participant C, lines ) B

And when the audio feedback came, there were examples to say You evaluated this when you did this and that, and it sort of, it sort of clicks, and you say, Oh, thats what they mean when they say critically analyse, and evaluate, and its just those simple things that sort of click. (Participant C, lines ) B

Other themes Inclusion – language; learning style Inclusion – language; learning style Practicalities, e.g. open plan environment Practicalities, e.g. open plan environment Emotional response Digital native v. digital immigrant

...the words that people use in writing is different from the words that people use when they say, and for me, I think, personally English not being my first language, I sort of get people more when they speak, because of the tone of the voice, and you know, I sort of understand the words a bit better than when I just read the words. (Participant C, lines 381-5) B

Well its something that I hadnt considered until... I started to open it, and he started to talk, and everybody else was in the office, what if its a really horrendous mark? (Participant B, lines 204-6) B

Tutors views More enjoyable Feel more exposed – at first Takes equal time – give more feedback in time available Technicalities (e.g. second marking, sharing feedback) more time consuming – at first Need to learn to keep feedback down to a reasonable length Practicalities, e.g. quiet place to mark

Future strategies Consider where to put the mark Encourage participants to listen with the assignment in front of them Keep participants informed, e.g. feedback release dates, when mark will be communicated Use with other modules – formative and summative Offer choice – text or audio (inclusive feedback) Research possible correlation between response to feedback and grade attained

The techy bit......

Hand-held digital recording device Demonstration One I prepared earlier Example feedback

Audacity Demonstration Available via

Other useful tools Any Audio Converter – to reduce file sizes / convert to MP3 – available via Wavepad Sound Editor – to edit files – available via

Application to practice

Practical guidance Consider using a combination of summative individual feedback and overview commentary Limit the length of recordings to avoid the danger of providing too much feedback Need for increased sensitivity in providing critical feedback by means of this medium (Ribchester et al, 2008; France and Wheeler, 2007)

More practical guidance Which device? Second marking External examiner Storage space Quiet space to mark Dissemination to students Feedback from students

Getting started.... Pilot with a small group? Use for formative feedback? Use in tutorials?

Discussion Working in pairs / small groups: Discuss the potential application of this approach to your practice What strategies could work in your environment? E.g. Whole group or individualised audio feedback? Formative or summative feedback? What would you need to do, in order to prepare students, staff and others? (e.g. External Examiner) Optional extra – have a go at producing some audio feedback

Finally..... Summary Comments / questions Thank you!

References Fleming, N. and Baume, D. (2006) Learning Styles Again: VARKing up the right tree! Educational Developments, SEDA. Issue 7.4., p.4-7. France, D. and Wheeler, A. (2007) Reflections on using podcasting for student feedback. Planet No. 18. [Online] Available from [accessed ] Goodyear, P. (2001), Effective networked learning in higher education: notes and guidelines: Networked Learning in Higher Education Project (JCALT) [online]. Available from: [accessed ]. HEFCE (2009) National Student Survey [online]. Available from: [accessed ]. Merry, S. and Orsmond, P. (2008), Students Attitudes to and Usage of Academic Feedback Provided Via Audio Files. [Online]. Available from [accessed ]. OBrien, R. and Sparshatt, L (2007), Mind the Gap! Staff Perceptions of Students Perceptions of Assessment Feedback [online]. In: Higher Education Academy Annual Conference, Harrogate, – Available from: [accessed ]. Race, P. (2008). Aligning assessment to current trends in HE. Staffordshire University, Ribchester, C., France, D. and Wakefield, K. It was just like a personal tutorial: Using podcasts to provide assessment feedback. In HE Academy Annual Conference 2008, Harrogate, – [Online]. Available from: doc [accessed ]. doc Rotherham, R. (2008a), Sounds Good: Quicker, better assessment using audio feedback. [Online]. Available from: [accessed ]. Taras, M. (2002), Using Assessment for Learning and Learning from Assessment. Assessment and Evaluation in Higher Education. 27(6) p Waterfield, J. and West, B. (2005) Staff-Student Partnership for Assessment Change and Evaluation (SPACE) Project. [Online]. Available from [accessed ].