1. Have we been getting feedback to students wrong? From conventional practice to strategies that make a difference David Boud University of Technology, Sydney University of South Australia, 12 November 2013

2. http://darden- united.blogspot.co.uk/2012/02/feedback-part- one.html

3. http://www.throughthetollbooth.com/2012/0 5/10/the-power-of-feedback/

4. http://www.xldatasoft.com/feedback.htm

5. usdgreeklife.blogspot.com

7. What is the problem? No matter what we do about feedback, it is always found wanting and students criticise us Is it just a matter of doing what we do now just with more vigor and more systematically? Is our use of the term feedback based on a misconception?

8. Some typical solutions Feedback more often means better feedback Quicker turnaround time means better feedback Automated feedback means better feedback

9. Conventional feedback Adjunct to marking Undertaken by teachers on students There is a vague hope that it might be taken into account But, no direct response is required or expected

10. If we didnt use the conventional approach, how might good feedback be created? Judge it in terms of effects Focus on what learners do, not what teachers do Examine key ideas from feedback in other disciplines Ensure that feedback is self-improving

11. What is feedback? a process whereby learners obtain information about their work in order to appreciate the similarities and differences between appropriate standards for any given work, and the qualities of the work itself, in order to generate improved work Boud and Molloy 2013

12. What does feedback do? It bridges the gap between teaching and learning, ensuring the curriculum is adjusted to the needs and learning of students It cannot be enacted without the engagement of participantsstudents and teachers. It only makes sense and it is necessarily stimulated by what students actually do.

13. Generations of feedback Feedback Mark 0 Conventional. Pre-feedback Feedback Mark 1 Behavioural. Closed Feedback Mark 2 Agentic. Open

14. Analysing the characteristics of feedback What are the features of different conceptions of feedback? How does each form operate?

15. Feedback Mark 0: Conventional Pre-feedback

16. Feedback Mark 0 Approach Conventional Locus Teacher initiated Features Taken-for-granted act of teacher/assessor Location At end of teaching sequence Effects Effects not detected directly Learner involvement No student involvement needed Information provided Information provided not influenced by effects Goal Study improvement Feedback loop None explicitly

17. Feedback Mark 1: Behavioural Closed feedback

19. Feedback Mark 1 Idea of feedback taken from control systems Information is taken from the student and used to influence their subsequent work This work is monitored to determine if the desired effects have been achieved Knowledge of effects is used to provide information to the same students and for later cohorts

20. What is essential in feedback Mark 1?

21. Feedback Mark 0Feedback Mark 1 Approach ConventionalBehavioural/cognitive Locus Teacher initiatedTeacher-driven Features Taken-for-granted act of teacher/assessor Closed system Classic feedback Tight loop Location At end of teaching sequence During learning Effects Effects not detected directly Effects closely monitored by teachers Learner involvement No student involvement needed Student involvement in response to specific stimulus Information provided Information provided not influenced by effects Information provided changes in response to immediate effects Goal Study improvement Task performance improvement Feedback loop None explicitlySingle loop

22. Problems with Feedback Mark 1 The teacher (or teaching system) is the driver and needs to continually provide comments and monitor the situation The learner is dependent on the teacher (or teaching system) to generate what they need to learn It is not sustainable assessment. It doesnt equip the student beyond the immediate task or course

23. What is sustainable feedback? That which doesnt continually need a teacher (or teaching system) to generate Helps develop a students judgement of their work Develops learners capacity to identify appropriate standards and criteria Develops learners ability to locate and access useful sources of feedback Involves learners working with multiple others in giving and receiving feedback

24. http://www.strath.ac.uk/learnteach/feedback/

25. Feedback Mark 2: Agentic Open feedback

26. Feedback Mark 2 Taken from open adaptive systems Central role for learners as active – two-way interactions/dialogue between giver and receiver – use of peers, non-human sources and practitioners as well as teachers – other parties used to enable learners to calibrate their own judgements

27. http://theteachingtomtom.wordpress.com/20 11/09/28/feedback-from-students- %E2%80%93-who-cares/

28. Feedback Mark 2 Feedback is a curriculum element responding to and driving learning – a pedagogical practice integral to all learning processes – deployed by learners as needed for their own learning paths – feedback becomes a design feature of courses, located to enable: sufficient practice to be had feedback loops to be completed effectiveness in self-judgement developed as a learning outcome

29. Feedback Mark 0Feedback Mark 1Feedback Mark 2 Approach ConventionalBehavioural/cognitiveAgentic Locus Teacher initiatedTeacher-drivenLearner-driven Features Taken-for-granted act of teacher/assessor Closed system Classic feedback Tight loop Open system Adaptive/responsive Location At end of teaching sequence During learningDuring learning and beyond Effects Effects not detected directly Effects closely monitored by teachers Effects monitored by teachers and learners Learner involvement No student involvement needed Student involvement in response to specific stimulus Student engagement intrinsic to process dialogic Information provided Information provided not influenced by effects Information provided changes in response to immediate effects Information provided changes in response to effects Goal Study improvement Task performance improvement Judgement performance improvement Feedback loop None explicitlySingle loopDouble loop

30. What to consider in adopting Mark 2 Talk to students about what feedback is for Construct students as active feedback seekers See feedback processes as a dialogue about what constitutes good work and how it can be achieved Respond to what students need As for all legitimate forms of feedback: check on the effects of what students can do as a result

31. Compare judgments Activity 2 Others judge work Student asks for specific feedback Student judges work Activity 1 Orientation to standards of work and purpose of feedback Plan for improved work Active role of students in eliciting, processing and using feedback

32. The feedback-enabled curriculum Has early strategies to shift learner identity to becoming self- regulated Positions feedback as part of learning, not as an adjunct of assessment Equips students to be skilled and comfortable with negotiating learning outcomes, feedback processes and information needs Fosters ongoing dialogue between students and teachers about feedback processes, the nature of standards and the practicing of judgement. Introduces activities to enable students to calibrate judgement (of their own work and that of others)

33. Curriculum features characteristic of Feedback Mark 2 FeatureExamples Learners orientated to the purposes of feedback Explicit learning outcomes relating to developing judgements and collaboration with peers, clear expectations that students actively participate in classes and that information received will lead to action Learners participate in activities promoting self-regulation Activities to build student engagement and foster self- regulation through self- testing of understanding, students re fl ecting on how the standard required compares to their execution of the task or planning what information they need to meet learning outcomes Learner disposition for seeking feedback is developed Development of feedback seeking skills through early practice activities including identi fi cation of appropriate criteria, formulating comments on others work, practice in identifying what kind of comments are needed on assignments Opportunities provided for production of work Opportunities for students to produce work of the kind that is central to learning outcomes through multiple tasks well designed for this purpose, not all of which might be formally graded Calibration mechanisms Channels to enable learners to check knowledge sources, develop understanding, calibrate their judgement against expert work and peer work, regular opportunities to judge their own work before it is marked Incremental challenge of tasks Development of sequences of tasks that progressively and realistically challenge learners, assessment tasks progressively build capacity to tackle more complex problems Nested tasks to allow for feed forward Timing and design of tasks to permit input from others (teachers, peers, practitioners and learning management systems, as appropriate) and self on each task, to be utilised to bene fi t performance on subsequent tasks Learner as seeker and provider Opportunities to practice giving as well as receiving of feedback. Orientation of learners to dimensions of the target performance (they need to engage with the desired learning outcomes, so they can make and articulate a comparative judgement)

34. Clarifying: what is this saying? Comments on students work are still vital But unless there is evidence of effects the person offering comments does not know how to calibrate their responses The opportunities for doing this are far less than the need for feedback, so other mechanisms are needed Unless we can find more ways to actively mobilise students to help themselves and each other, then courses will not be sufficiently tailored to meet their needs

35. Guidance for those offering comments Be wary of the old nostrums Involve the learner at every stage – if they are positioned as a passive recipient that will act as such Think about what you want to influence

36. Hatties model for feedback comments Comments can be directed at four different levels of operation of the student. Feedback will be ineffective if directed at an inappropriate level. The responses of students and their efficacy are dependent on the focus and type of comments they get. If the focus is inappropriate to their needs, feedback may be ineffective, because the student is unable to transform information into action where it is needed most. Hattie and Timperley 2008; Hattie and Gan, 2011

37. Levels of operation at which comments are pitched: Task focused Process focused Self-regulation focused Person focused

38. Levels of operation at which comments are pitched: Task focused – Most common Process focused – More effective Self-regulation focused – Most needed Person focused – Mostly ineffective

39. Elements of self regulation focus capacity to create internal feedback. ability to self-assess. willingness to invest effort into seeking and dealing with feedback information. degree of confidence or certainty in the correctness of the response. attributions about success or failure. level of proficiency at seeking help.

40. http://www.justinbuzzard.n et/2012/04/03/leaders- need-feedback/

41. Some useful websites on feedback University of Edinburgh: http://www.tla.ed.ac.uk/feedback/index.html http://www.tla.ed.ac.uk/feedback/index.html University of Strathclyde: Feedback is a dialogue http://www.strath.ac.uk/learnteach/feedback/ http://www.strath.ac.uk/learnteach/feedback/ Re-Engineering Assessment Practices in Scottish Higher Education http://www.reap.ac.uk/ http://www.reap.ac.uk/ Assessment futures http://www.assessmentfutures.com http://www.assessmentfutures.com University of Hong Kong: Exploring the Feedback Conundrum http://hkufeedback.blogspot.com/

42. http://www.allthingsworkplace.com/2012/05/ what-why-and-how-feedback.html

43. What is the role of the digital environment? It offers some affordances, but without thinking differently about what feedback does, it just means doing bad feedback more efficiently.

44. Examples of the role of the digital environment for Mark 2 feedback For students, it can offer: – More practice – More/different occasions for practice – Quick knowledge of results/ calibration of judgements – Remedial sequences instantly at hand For teachers, it can offer: – Instant records of prior feedback data and student responses to it For both, it can offer: – More opportunities for dialogue on standards and judgements – Management of self and peer feedback

45. An exampleRe:View ReView is a web application developed to aid marking, feedback and graduate attribute development. Student self-assessment and comparisons with tutors is an option that can be selected for each task It enables students to track their development over time http://reviewsecure.com

46. Close up of staff marking screen with students self assessment

47. http://theheroineaddicts.blogspot.co.uk/2010/ 09/feedback.html

48. References Boud, D. and Molloy, E. (2013). Rethinking models of feedback for learning: the challenge of design, Assessment and Evaluation in Higher Education, 38, 6, 698-712. Boud, D. and Molloy, E. (Eds) (2013). Feedback in higher and professional education. London: Routledge. Hattie, J. and Gan, M. (2011). Instruction based on feedback. In Meyer, R.E. and Alexander, P.A. (Eds) Handbook of Research on Learning and Instruction, New York: Routledge. Hattie, J. and Timperley, H. (2008). The power of feedback, Review of Educational Research, 77: 81-112. Jolly, B. and Boud, D. (2013). Written feedback: what is it good for and how can we do it well? In Boud, D. and Molloy, E. (Eds) Feedback in higher and professional education. London: Routledge, 104-124. Shute, V.J. (2008). Focus on formative feedback. Review of Educational Research, 78:153- 189.