1. Improving student engagement and learning through examination feedback Carole Sutton, Jo Sellick, Kim Stevenson, Jonathan Clark, Jane Collings and Vehid Salih

2. Inclusive exams and feedback does not mean ‘easier’ or ‘avoiding things’  Formative exams  Staging of exams ( progressive & building complexity)  Feed-in Feed-in Exams set up, marking criteria, briefing, practice, preparation Feed-forward Formative ‘feedback’ on practice papers, discussion, peer feedback against marking criteria, self -assess Feedback group, workshop, generic, written, verbal, mp3, marks/grades Learning Space

3. Background literature The challenge to the HEI sector: – Examination marks are generally lower than for other forms of written coursework assessment – Exams cause particular concern to the student especially where the examination mark makes a significant contribution to the degree classification (Bridges et al. 2002, Simonite, 2003) ‘The aim for exams as well as other forms of assessment should be to engage students in dialogue about feedback and assessment, which enables students to take part in a discussion that demystifies the process of assessment … feedback need not be exclusively summative, even when the form of assessment is by examination’. (Blair et al, 2013:1051)

4. Background literature 2: Havnes (2004) argues exams assess surface learning and have a negative and limiting impact on student learning Carless (2014) contends feedback should come immediately after the assessment so students receive feedback whilst content is fresh in their memory.

5. Policies across the sector (snapshots) University of Birmingham All Students (excluding yr 3) – Generic feedback on each examination question within an assessment (e.g. essay style or numerical problems) or for the assessment as a whole (e.g. MCQs). – It should be provided within ten working days of the publication of results. – Schools may provide individual feedback in place of generic feedback to all students who have undertaken the assessment, not just those that request feedback. – Students who have failed exams should be offered additional feedback

6. Newcastle University Feedback and marks to be provided on all exams within 20 working days from the end of the exam period. For Semester 2 exams, feedback can be provided either within the 20-working-day deadline or at the start of the next term/semester. Minimum, whole-class (generic) feedback must be issued to students within these deadlines, and students must be informed that they have the option to request individual feedback. No requirement for students to see or have returned copies of scripts to students or allow students to see scripts, although they may choose to do so. However, they do have a legal right to request access to the breakdown of marks and markers’ comments.

7. Northampton University ‘Generic feedback on written examinations will be available to students if requested. Some programmes may also choose to give those students who specifically request this sight of their scripts under supervision by a member of staff’.

8. At Plymouth NUS benchmarking tool The general lack of feedback to examinations has prompted student concerns (NUS, 2009) – it creates uncertainty and limited opportunities to improve and learn, particularly for any future examination based assessment. UPSU – QAA Student Written Submission 2016 R16: We strongly recommend the University looks into Examination Feedback as recommended by the NUS benchmarking tool

9. TFAS Examination Feedback project Carole Sutton & Jo Sellick, Faculty of Business Methods Literature Review informed the development of a Toolkit – Pre-survey student experiences – Exam feedback model selection – Examination feedback models & implementation – Post-survey student experiences Feedback and next steps

10. Exam Feedback Toolkit (interim draft) Key Factors in Choosing a Model of Examination Feedback Across the Programme – Exam type & purpose of the exam (linking to ILOs) Cohort Size Location & Type of Student Timing and time scales Access to exam scripts Within a module – Formative and Summative assessment approach

11. Examples of feedback to formative and summative examinations and testsSuggestions how to Generic written Summary covering minimum requirements and linked to learning outcomes – could be based on distinguishing between classifications Email Moodle ‘Model’ answers and/or exemplars bank that can be accessed by students on-demand Generic verbal/ audio Q&A session Timetabled session (note potential of content capture) Moodle Peer discussion/ feedback (e.g. of individual formative and/ or exemplars) Podcast/ video Individual written Template/ summary feedback form with minimum requirements and linked to learning outcomes with specific reference on how to improve/ what to do next Hard copy (collected from/ distributed by tutor/ support office) Moodle PebblePad Email Individual verbal / audio 1:1 appointment with exam marker using script (formative mock and/or summative) Generally promoted with on-demand appointments Blocked out periods for appointments (‘exam open day’/ ‘feedback week’ etc.) Moodle PebblePad Email Meeting with Personal Tutor who has received reports from markers as part of PDP process Examination ‘open day’ to look through papers with tutors Individual audio recording such as podcast

12. Case Study Sociology Module: Stage 3: Social and Political Theory. Semester 1 (13 weeks) Assessment: Seen exam paper essay style (2 questions in 2 hrs) Formative Feed-in: 1. Dec 18, 2015 – Exam release & talk through plus Q&A 2. Jan 14, 2016 – Assessment criteria, critical thinking and exam revision scrutiny plus 5-minute outlines & formative feedback (Group A) 3. Jan 15, 2016 – Assessment criteria, critical thinking and exam revision scrutiny plus 5-minute outlines & formative feedback (Group B) Summative Feedback: Feb 16, 2016 – Group Feedback, exam script review, invitation for 1 to 1 feedback plus NSS session.

13. Case Study: Sociology Staff Challenges: To sensitively balance exam script return with the need for grade privacy when conducting group feedback To effectively manage the additional time required should many or all students request individual 1 to 1 feedback meetings To ensure feedback is as useful as feed-in. Establishing a clear, concrete link between exam feedback and on-going PDP should help here. Measurable Results: Acad Year2008/092009/102010/112011/122012/132013/142014/152015/16 Avg Exam Mark 53.84%44.29%49.50%56.95%54.16%53.42%63.33%65.79%

14. Preliminary feedback from students Students reported: – The pre-exam preparation tasks allowed me to understand how the exam was assessing my learning in relation to the MLOs (73.3% strongly agreed; 26.7% agreed) – I applied the exam preparation techniques (60.0% strongly agreed; 33.3% agreed) – I understand how the feedback comments related to the exam mark awarded (60.0% strongly agreed; 26.7% agreed) – I have been able to identify how to improve my examination and study techniques for the next exam (60.0% strongly agreed; 40.0% agreed) – The approach taken in the module has improved my confidence in undertaking exams (73.3% strongly agreed; 20.0% agreed) Student open comments: – “I think that exam preparation, guidance and feedback has progressed and improved throughout my 3 years here” – “I was given enough time to prepare for the exam through the questions being released early” – “This year it's been good, the exam session would be good for 2nd years who have lots more exams” – “In general it has been clear and concise process to take exams” – “Exam prep was better for Sociology than Psychology”

15. Case Study: Law Module: Stage 1, Law Immersive (4 weeks) Assessment: in-class test (MCQ style and short answer questions) Formative Feed-in: Dedicated session end of week 2. 30 exemplar questions used for mock test Tutor works through correct answers in interactive session Revision Techniques: dedicated session with practice questions, exercises and ideas on how to revise and what to expect Summative Whole cohort generic verbal feed-forward (w.b. 1 st week Nov) Areas for improvement, especially for the resit Individual 1:1 meeting student & tutorial tutor Go through exam paper & markers comments

16. Case Study: Law Student Achievement 88% passed 1 st attempt; (93% after resit) 42% gained first; 24% 2:1 Mark range 18 – 96; average 62.02% Staff Challenges Practical ie timetabling 1:1 slots and scheduling in relation to resit DAS

17. Preliminary feedback from students Students reported engaging in a range of preparation, formative assessment/feedback opportunities, including – Exemplar exam questions (62.7%) – Q&A session in a classroom (67.2%) – Examination Technique workshop/training (35.8%) – In-class workshop on model examination Q&A (31.3%) – Mock examination (32.8%) Students engaged in a range of individual revision activities, including – Own revision (94.0%); Online resources (50.7%); Use of Learning development materials (32.8%); Mock exam questions (31.3%)

18. Semester two modules (planned) Stage 2 Law (core) modules Module 1: – Exam format: mix of MCQ, short answer, short essays. – Feed-in: Student grade exemplars; outline answers; submit individual test essay with individual feedback – Feedback proposed: 1:1 written using a bespoke proforma

19. Emerging themes from the TFAS project so far … Exams are just another form of assessment that students should expected feedback on to understand and improve their performance Logistical considerations – Retention of the exam script does not prevent feedback – Constrained by exam timetable » Requires adaption of feedback to meet the time constraints No equivalence of e-submission for exams » Challenges for feedback, especially Semester 2 Staff workload allocation – adjustments to align to other forms of assessment

20. Vehid Salih Plymouth University Peninsula Schools of Medicine and Dentistry

21. Integrated Dental Science Clinical Dentistry Professional Development Goal = Safe Beginners 58 places per year (Total School 338) Special Study Unit MCQs Critical Thinking Progress Tests Unseen case Audit project Teamworking Progress Tests ISCE Form Ss and Ts Case Presentation Specialist Visit Reports Progress tests Special Study Unit MCQs Inter- Professional Engagement

22. NSS sub-sections 201420132012 Teaching Quality (Q1-4) 999692 Feedback & Assessment (Q5-9) 917863 Academic Support (Q10-12) 978881 Organisation and Management (Q13 -15) 948258 Student Feedback NSS Dentistry 2014

23. NSS sub-sections 201420132012 Learning Resources (Q16-18) 999286 Personal Development (Q19-21) 999185 Overall Satisfaction 989682 Sector Average Scores 968778 Student Feedback NSS Dentistry 2014

24. NSS Rankings 2015 (Dentistry) Average positive NSS score 2014 – 96.71%

25. Clinical data collection  Proficiencies, Form S targets Professional performance feedback/reflection Progress Tests ISCE Multiple Choice Questions SSU Reports All Clinical Data collected at chair side with immediate clinical and professional performance feedback and reflection (recorded on Assessment Database). Types of Assessment

26. Next steps and what now for you?

27. References Blair, A., et al. (2013). "Can dialogue help to improve feedback on examinations?" Studies in higher education 39(6): 1039-1054. Carless, D. (2014). "Exploring learning-oriented assessment processes." Higher Education. 69(6): 963-976 Havnes, A. (2004). Examination and Learning: An activity- theoretical analysis of the relationship between assessment and educational practice. Assessment and Evaluation in Higher Education 29(2): 155-164 Teaching & Learning Assessment and Exams Resource https://www.plymouth.ac.uk/your-university/teaching-and- learning/guidance-and-resources/assessment https://www.plymouth.ac.uk/your-university/teaching-and- learning/guidance-and-resources/assessment