1. Satisfied – but students want more feedback: Why did I get 37%? 1 University of Exeter Learning and Teaching Conference, May 22 nd 2008 Professor Brenda Smith Higher Education Academy

2. Aims… Emphasis assessment for learning rather than of learning Encourage the active engagement of students in their learning Enhance assessment and feedback process for both staff and students

3. Road Map of the Presentation Myths about assessment Burgess Report – a case for change Assessment methods – have we got it right? NSS tables in relation to Exeter Focus on feedback

4. John Cowan, former Director, OU, Scotland Assessment is the engine that drives learning

5. 5 1.Certification 3. Learning 4. Sustainability 2.Quality Assurance Why do we assess?

6. Whats wrong with assessment? The evidence: QAA Subject Reviews National Student Survey Rising concern about cheating and plagiarism Student complaints Some issues: Too much summative and insufficient formative assessment Insufficient effective feedback that comes too late Variations in practice Unscholarly practices (especially with numbers) Myths and traditions that inhibit change

7. Myths… Students benefit from jumping assessment hurdles unaided Experienced tutors apply consistent assessment standards Students dont want feedback on exams That it is possible to distinguish the quality of work to a precision of one percentage point Moving back to exams will solve cheating problems Double marking will ensure fairness and reliability Assessment standards are consistent across an institution

8. The Burgess Report

9. The Burgess Report A case for change Present system cannot capture achievement in some key areas of interest to students and employers Emphasis on the top two degree classes wrongly reinforces an impression that a Lower Second or Third Class Degree is not an achievement Institutional methods for calculating the degree classification could be clearer

10. The Burgess Report A case for change A summative system – is at odds with lifelong learning There is a need for greater emphasis on additional information that is currently contained in the European Diploma Supplement and academic transcript (an official record of a learners programme of study, grades achieved and credit gained) Institutional practices – complex and inconsistent

11. Complexity, variety and inconsistency of classification assessment regulations (SACWG) surveyed 35 HEIs 25 used percentage marks; eight used grades; and two used both percentages and grades 18 institutions aggregated or averaged percentage marks to arrive at the honours degree classification; 4 used a profiling approach; and 13 used both or either SACWG: Student Assessment and Classification Working GroupStudent Assessment and Classification Working Group

12. Complexity, variety and inconsistency of classification assessment regulations Variations in degree classification can be caused by: Differing institutions award algorithms Dropping a no of modules when calculating the degree classification A shift towards course work and away from formal examinations Institutions diverged on their approaches to borderline performances Variations in the weightings given to the penultimate and final years of full-time study Ref: SACWG and Mantz Yorke et al

13. Relative weightings given to results at Levels 2 and 3 Ratio, Level 3:level 2Frequency 4:11 3:14 70:303 2:14 3:22 4:31 1:15 Ref: Mantz Yorke et al

14. A single index of performance, whether it is a grade-point average or an honours degree classification, tells nothing on its own about the trajectory of a students development. It does not differentiate between the student who progresses steadily over the course of a programme and the student who starts slowly but in the final year produces work of an outstanding standard Issues involving student trajectory

15. The student voice - what they would like… More advice & support Relevant & real-life topics Choice of assessment topics Variety of assessment methods Feedback to help me learn For the assessment load to be spread out Clear criteria MESA Project I never realised that assessment was for learning

16. Department A

17. Department B

18. Department C

19. Department D

20. Assessment Methods at Exeter? University of Hertfordshire analysed assessment methods across the institution

21. Tasks Multiple choice Open book IT based Interactive Group element Role play Medium Oral Numeric Diagram/Pictori al Cognitive skills Analytic Evaluative Skills focus Primary research Theory focus Time-span Reflective Process/Periodic Portfolio Who assesses? Self assess Peer assess Self set element Work-related Practice focus Case Study University of Hertfordshire

23. National Student Survey (Full-time students UK) The Assessment and feedback scale2006 2007 1. The criteria used in marking have been made clear in advance 67 68 2. Assessment arrangements and marking have been fair 73 72 3. Feedback on my work has been prompt 51 4. I have received detailed comments on my work 55 54 5. Feedback on my work has helped me clarify things I did not understand 51

27. Video Man Friday Please watch the video and ask yourself are you always clear about the criteria you set for students? How do you ensure that they understand what is required of them? Could you give students practice in setting and then marking against criteria?

31. Comparison of courses within x University in terms of students use of feedback Assessment Experience Questionnaire (AEQ items) http://www.open.ac.uk/science/fdtl/overview.htm Best course Worst course % strongly agree % strongly agree The feedback helps me to understand things better36%6% The feedback shows me how to do better next time31%4% The feedback prompts me to go back over material covered earlier in the course 13%1% % agree strongly or agree I do not use the feedback for revising17%44%

32. Everyone can make a difference.. What are the positive features of your module/course? How do you: Identify them? Disseminate them within departments? Disseminate across the institution? Disseminate externally? Draw on effective practices in learning and teaching from external sources? Embed that practice?

33. Dissemination 1. For awareness 2. For understanding 3. For action. By action here we mean some change of practice inspired by, and making use of, the materials or processes produced by your initiative 4. Evaluate the success of your dissemination

34. Conditions necessary for innovations to improve Learning, Teaching & Assessment Implement in student focused ways Departments that value teaching Workload that are not too high Effective leadership of learning, teaching & assessment Perceived recognition and rewards

35. Recommendations in relation to support, adaption & implementation 1.Recognise in workload allocation formulae 2.Recognise scholarship of teaching outcomes – publications etc 3.Provide small scale grants 4.Provide sources of institutional support and advice 5.Encourage development of cross institutional networks & events 6.For academic developers to work alongside project teams 7.Ensure time release as appropriate Ref: Carrick Institute, Australia

36. Managing the Assessment process FEEDBACK TO HELP STUDENTS TO LEARN

40. Why feedback? What kind of feedback do learners need? How much is enough? How often is often enough? How can we make feedback efficient and effective? How can we ensure feedback is taken seriously? Should feedback be one way? Can students give each other feedback?

41. Why give feedback? Increase motivation and interest (Give choice) Improve performance Eliminate misconceptions Develop independence Raise confidence Create trust Individualise learning Develop a learning community

42. Paradoxes of feedback Teachers complain about it Students ignore it They get it too late When they get it its not useful

43. Questions….. How many hours do you spend giving feedback? At what time in the semester do you give most feedback? Could you give more feedback upfront? Could feedback be spread out throughout the year? Could you encourage peer feedback? Could you give feedback in different ways (MP3)?

44. Staff comments on feedback and marking They often dont even collect their essays…..Some of them are more ready to argue about their mark than they are to collect their essay They came up to me afterwards…and they said well we didnt really understand what you meant by evaluate. And I kind of thought well isnt it obvious, but the more I think about it I wonder if it is actually obvious to students, what the word means Ref:www.assessmentplus.net

45. 45 Do we try and trip our students up?

46. Evaluation of tutor feedback omission Use of English Ask for clarification Error Give clarification Praise Encouragement Feed forward -ive 30% 20% 17% 11.5% 10.5% 3.1% 1.5% 0% 0% OU/SHU Improving the Effectiveness of Formative Assessment in Science

47. A study (Carless 2006) reported a substantial difference in the perceptions of tutors and their students on feedback on assessed work A study at the University of Sydney, ran focus groups with 5,000 students to gain an in-depth understanding of student perceptions of feedback 3 dimensions emerged Differing perceptions (most recognisable if written) Impact of feedback – timeliness, significance & 1 st year experience (difference between University and School) Credibility of feedback – this was related to the students perceptions of the lecturers themselves Do you know what your students think about feedback?

48. Students comments They are not very specific when they mark. I got some good evaluation, what does that mean? You put so much effort into this work that you really want feedback and to know they have read it properly. If you just get a mark you think, Ive worked for days and thats it? I think it would be helpful as well if when we had our feedback we could relate it to the assessment criteria Ref:www.assessmentplus.net

49. 49

50. Do these comments aid learning? A Excellent C- Fine C/C- Interesting C Opening lousy - main part OK C- Your writing is appalling D Terrible - didnt you learn anything at school about essay writing!

51. Comments difficult to understand… This essay is not sufficiently analytical This report is not logically structured However we also need to focus our efforts on strengthening the skills of self-assessment Students need to be active in this process We need to use language that they can understand How might we use students in this process?

52. Involve the students Feedback as a dialogue For Student Completion 1. These are areas of my work that I think are good for the following reasons 2. Please comment on the following areas of work 3. What I want to improve or do differently next time 4. The mark I think this piece of work deserves is For Staff Completion

53. Order of giving feedback 1.Good news – what was done well 2.Enhancement news – what still needs to improve 3.Options – what can be done to improve 4.Plans – what the learner intends to do 5.Commitments – what both parties agree to do, how, to what standard and by when?

54. Formative feedback - a challenge to course teams Is the feedback motivating and does it aid learning? Do students use feedback to improve future performance? Do you and the student track the changes? How can you improve the need for students to want feedback and use it as feed forward? Can you give credit to students who use their feedback?

55. How can we engage with our students? Engage from day 1 Rethink induction Get students involved in setting and marking against the criteria Encourage peer feedback & peer assessment Rethink the course committee agenda Ask their views The one minute paper Structured feedback session…..

56. Structured Feedback Session This is an activity to give an individual and collective voice to your experiences of assessment and feedback TASK Individually think of all the good things about your experiences of assessment and feedback Write each one on a separate post-it note In your group put all your post-its on a flip chart and say them out loud as you post them up Now collectively agree on the top 3 aspects of your course in relation to assessment and feedback that rate the highest

57. Structured Feedback Session TASK Individually think of all aspects of your assessment and feedback experiences that could be enhanced Write each one on a separate post-it note In your group put all your post-its on a flip chart and say them out loud as you post them up Now collectively agree on the top 3 enhancements that could be improved Taking those top 3 aspects lists some ideas that could be taken forward

58. Applying the seven principles of assessment

59. Seven principles of good feedback practice 1.Facilitates the development of self-assessment (reflection) in learning 2.Encourages teacher and peer dialogue around learning 3.Helps clarify what good performance is (goals, criteria, standards expected) 4.Provides opportunities to close the gap between current and desired performance (SENLEF Project)

60. Seven principles of good feedback practice 5.Delivers high quality information to students about their learning 6.Encourages positive motivational beliefs and self- esteem 7.Provides information to teachers that can be used to help shape the teaching Ref: Juwah, Macfarlane-Dick, Matthew, Nicol, Ross & Smith (2004 )

61. A vision of students

62. Do league tables give us the whole picture? Does our position on the league tables guarantee a really positive learning experience for all our students? How might we collectively work towards an enhanced student experience?

63. Financial indicators Job prospects Academic indicators Ease of entry and demand Student life indicators 1 Lampeter, University of Wales 2 Heythrop College, University of London 3 Stirling University 4 School of Pharmacy, University of London 5 University of St Andrews 6 St George's, University of London 7 Harper Adams University College 8 Bishop Grosseteste University College 9 King's College, London 10 University of Edinburgh 113 University of Exeter

64. Congratulations EXETER is a top 20 UK University, according to all four of the main higher education league tables BUT does that guarantee that the student experience is enhanced? HOW can you make that student learning experience even better?

65. Reflections …..