1. Using group discussion to enhance evaluation of clinical teaching: A pilot study Dr Charlotte Mackay –GP ST4 Medical Education Fellow, Queen Mary University of London Maria Hayfron-Benjamin –Lecturer in Medical Education, Queen Mary University of London

2. Background Varying attitudes towards quality assurance processes Teachers view process as an imposition and distraction (Booth et al., 2008) Students report ambivalence about whether their views and opinions are taken seriously (Spencer and Schmelkin, 2002)

3. Where we are currently at QMUL What was good about this placement? “ Everything” What did you find useful about this teaching? “Nothing” “Clinical teaching is good” Was there something you thought should be changed to improve it? If so, what would you suggest? “Not really”

4. What we stand to gain from improved evaluation and feedback For tutors: Valuable information on teaching methods –what worked/did not work and why? For students: Development of skill set to use throughout careers for teaching, continuing professional development and appraisal For faculty: Highlight opportunities for curriculum development

5. QMUL Pilot: Methods Cohort study Medicine in Society GP Placement 55 year one students (groups of 6-8 per practice) Eight GP tutors (and eight community tutors)

6. QMUL Pilot: Timeline October 2014 Students and tutors recruited into study by email November/December 2014 Initial practice visits to introduce project in more detail and discuss evaluation template February/March 2015 Student group discussion (observed by MHB and/or CM) and completion of feedback proforma Evaluation sent to GP tutors once MedSoc module completed April 2015 Focus group interview with students GP tutors invited to feedback via email

8. Results: Observing student group discussion Discussion lasted 45-60 minutes Student chair and scribe Lively debate Range of voices heard

9. Results: Focus Group Three participants Core themes identified

10. Focus Group Theme: Benefits of group discussion “Sometimes you want to say something but if you have to fill in a form you forget it, but if you discuss it with others and they come up with an idea, then you’re like “oh yeah”’ “It was something fun. It wasn’t something that you write it down and you find it boring” “I felt we worked better as a group, finding things to highlight and things that I wouldn’t really pick up on by myself” “It makes you appreciate things about the teaching that you might not necessarily have noticed”

11. Focus Group Theme: Potential pitfalls of group discussion “Some people expressed to me by themselves like “Oh I don’t really like that or this” but when we were in the group setting ….they kind of tagged along, so they weren’t as genuine as if you had asked them by themselves”

12. Focus Group Theme: Loss of anonymity “I don’t mind it being anonymous or not, its constructive feedback…it’s something more to improve” “As long as you’re polite then I don’t think it should be a problem at all”

13. Focus Group Theme: The role of the chair “Our chair was really good. He was like “what do you think about this?”…and then around the group so that all the voices were heard…he organised everybody and if someone was just saying “I agree” then he made sure they were first to talk at the next point”

14. Results: Tutor Feedback Comments from two GP tutors and one community tutor GP tutors: Mixed response Recognised that comments had assisted their own personal development Community tutor: ‘Enlightening’

15. Suggestions for improvement Students: More detailed explanation of project needed beforehand Tutors: Proforma more explicit about specific MedSoc learning objectives

16. Discussion Broadly positive response from participants Students: More enjoyable than traditional feedback methods Benefits of group discussion to stimulate new ideas and clarify existing ones Tutors: Concern about time required to implement Recognise advantages of more descriptive feedback

17. Next steps Extension of pilot over 2015/16 academic year All MBBS Year1 and GEP MedSoc students to be involved

18. References BOOTH, J. COLLINS, S. HAMMOND, A. 2008. Considered evaluation of clinical placements in a new medical school. Assessment & Evaluation in Higher Education, 34, 17-29. CURRAN, V. CHRISTOPHER, J. LEMIRE, F. COLLINS, A. BARRETT, B. 2003. Application of a responsive evaluation approach in medical education. Medical Education, 37, 256-266. GMC 2011. Tomorrow's Doctors. General Medical Council. HAMMOND, A. COLINS, S. BOOTH, J. KALIA, S. 2009. Learning from Evaluation: A descriptive, student-informed approach. The Clinical Teacher, 6 (2) 73-78. NARASIMHAN K. 2001. Improving the climate of teaching sessions: The use of evaluations by students and instructors. Quality in Higher Education, 7, 179-190. RAUPACH, T., SCHIEKIRKA, S., MUNSCHER, C., BEISSBARTH, T., HIMMEL, W., BURCKHARDT, G. & PUKROP, T. 2012. Piloting an outcome-based programme evaluation tool in undergraduate medical education. GMS Zeitschrift fur medizinische Ausbildung, 29, Doc44-Doc44. SPENCER, K. J. & SCHMELKIN, L. P. 2002. Student Perspectives on Teaching and its Evaluation. Assessment & Evaluation in Higher Education, 27. STAKE, R. E. 1976. EVALUATE AN ARTS PROGRAM. Journal of Aesthetic Education, 10, 115-133.