1. Workplace Based Assessment for GP Specialist Trainees in hospitals: how well does it work? Abby Sabey, Senior Lecturer, UWE Michael Harris, Associate Postgraduate Dean, Severn Deanery

2. Background and aims How well does GP WPBA work in day-to-day hospital practice? What are the views of GP trainees (phase 1) and their hospital assessors (phase 2)? Joint UWE/Severn study with, & funded by, Severn Deanery.

3. Summary of the study 1.Phase 1: Mixed methods including: –questionnaire survey of GP STs in year 1 and 2 in two training locations within Severn (n=52, 67% response) to give overview –focus groups with subset of this cohort to develop insight (n=22) 2.Phase 2: Qualitative approach using: –semi-structured interviews with hospital assessors (those with GP STs) from three centres (n=15)

4. What did trainees say? They value the opportunity for a face-to-face discussion with a senior colleague; relationships are important Verbal feedback is useful but assessors may not be good at it – ‘cursory and superficial’, ‘too little too late’

5. What did trainees say? Briefer assessments are less useful than more in-depth methods such as multi-source feedback which generates responses from several co-workers Scores are valued less than comments Trainees avoid challenging cases in assessment, although these are the ones they learn from most in practice; –the system allows them to do this.

6. Most important messages Honesty in feedback is lacking this undermines the value and usefulness of WPBA Bias arises from choosing assessors – ‘getting your mates to do it’ weakens the value placed on assessments is added to the bias from choosing less challenging cases on which to be assessed.

7. Most important messages Just over half find it useful as a learning tool much scope for improvement in shifting the emphasis from assessment (‘ticking boxes’) to learning through quality, in-depth feedback The majority of trainees do not think it will make them a better doctor, and do not find it useful or valid.

8. What did consultants say? Assessors like the ‘mandate’ to give feedback But there are concerns about formal use of assessment records as ‘evidence’ Wide support for the qualitative elements of WPBA, ie the verbal and ‘free text’ areas –this balances the scores and tick boxes, helping to make WPBA a formative and developmental tool.

9. What did consultants say? Assessors also find it ‘difficult’ and ‘uncomfortable’ to –give feedback that is negative; –balance honesty with being a supportive educator.

10. Most important messages Verbal, qualitative elements bring a crucial depth to the process of assessment But honesty can be lacking in feedback, reducing the perceived and actual value of the process Consultants perceive WPBA is a poor discriminator and focused on a minimum standard of competence at the expense of recognising excellence –this loses an important motivation among trainees.

11. Most important messages Better links across hospital and GP supervisors could –underpin good practice, –help identify high-achieving trainees, –in turn boosting motivation.

12. Conclusions WPBA generates considerable volumes of work for all involved, but is failing to deliver high quality feedback that doctors find useful The lack of honesty, leniency and a ‘tick box’ mentality in assessment are now more widely acknowledged; a change in the system is needed to improve credibility and outcomes WPBA needs to encourage a more developmental culture within assessment to maximise educational value and help deliver better doctors.

13. Impact - what has happened since? Findings presented to RCGP who implemented some changes to the forms used to make them more relevant Published in Education for Primary Care –Sabey A, Harris M. (2011) Training in hospitals: what do GP specialist trainees think of workplace-based assessments? Education for Primary Care, 22(2), 90-99. Findings have fed into new plans in Switzerland for a similar system of GP training and assessment.

14. Discussion points How much do these findings resonate with your own experience? What should be done to make the WBPA system better? What can you do to improve WBPA practice in our own posts… –in hospital? –in General Practice?