1. Enhancing Assessment – Experience from psychiatry Dr Andrew Brittlebank Associate Dean, RCPsych

2. What I’ll cover… Psychiatric culture Modifying the culture The results so far

3. “Competition can make people wary of assessment, and efforts to provide feedback on progress and attainment can unintentionally be seen as threatening...WPBA requires a change in culture” PMETB/AoMRC, 2009

4. The pre-MMC culture in psychiatry Supervision – Embedded Apprenticeship ‘see one, do one, teach one’? Assessment – High stake

5. Supervision Mandated since the 1970’s Inspected at college visits Variable degrees of compliance Embedded in psychiatry

6. Read about one, do one… “In Europe, being a psychiatric patient is a bit like being a passenger of a pilot who learned to fly by reading about it” Compernolle, 1998

7. High stake assessment MRCPsych examination – MCQ, essay, long case Supervisor report Appointment committees

8. Modifying the culture Patient safety MMC and training WPBA assessors Psychiatric skills should be coached

9. Patient safety Conversations with patient safety and experience teams – “How do you know your doctors are safe?” NHSLA – “It’s the economy, stupid”

10. Training in WPBA – since 2006 Reforms in medical education – Shift away from traditional apprenticeship Purpose of WPBA – Formative Tools The standards – Develop performance dimensions Observation – Assess what you see and what you hear Frame of reference – Peer discussions

11. Teaching psychiatric skills “Watching your trainee is a bit tiresome” People respect what you inspect – Renewed interest in teaching and learning psychiatric communication skills – Cascading examiner training – WPBA guides for Core and Advanced training

12. Results so far Participation Reliability Impact

13. Participation rates GMC Trainers’ Survey 2010 – Psychiatry WPBA

14. Reliability data for CbD Based on GeneralisabilityTheory Psychiatrists assess reliably

15. Impact Mixed picture Depends on the question and how it is asked

16. Impact of WPBA – qualitative Added burden for trainees: “It’s a hassle, getting people to do the assessments” Assessment drives learning: “It’s good to know where you stand, so you know what you have to do” As a tool for reflection: “It has been helpful to reflect on how much you think you’ve improved”

17. Impact of WPBA - quantitative Julyan, 2009 – May enhance effectiveness of supervision Menon et al, 2009 Concerns about the tools and practicalities Babu et al, 2009 – Positive view of training in WPBA – Mistrusted by trainees Menon et al, 2012 – Trainers more positive about WPBA’s than trainees – But trainers surveyed six months after trainees (2009)

18. In conclusion The introduction of WPBA demanded a change in practice and culture It is work in progress It takes time – five years minimum

19. Thank you! Questions?