1. V0.4 SUPERVISION WORKSHOP CBFP – PART 2 RANZCP Fellowship Regulations 2012

2. V0.4 2 Contributors Dr Felicity Plunkett Director of Training, NZ Auckland Dr Wayne de Beer Deputy Chair Board of Education, NZ Hamilton Dr Martin Cohen Director of Training, NSW Newcastle Dr Neil Port Supervisor, NSW Newcastle

3. V0.4 3 Workshop program AM session 9.00 – 10.30 amCBFP training Professional Presentation (PP) Observed Clinical Assessment ( OCA) 10.30 – 10.45 am Break 10.45 – 12.15 pmEntrustable Professional Activities (EPAs) Mid and end of rotation Supervisor Assessments 12.15 – 12.30 pmStructures, boundaries & wrap up

4. V0.4 4 Workshop program PM session 1.30 – 3.00pmCBFP training Professional Presentation Observed Clinical Assessment ( OCA) 3.00 – 3.15pmBreak 3.15 – 4.45pmEntrustable Professional Activities (EPAs) Mid and end of rotation Supervisor Assessments 4.45 – 5.00pmStructures, boundaries & wrap up

5. V0.45 Workshop Objectives Recall the key elements of CBFP training (from Part 1 of this workshop) Understand and be able to use the other new Workplace-based Assessments: –Professional Presentation –Observed Clinical Activity (OCA) Understand and be able to evaluate whether trainees have achieved the summative assessments: –Entrustable Professional Activities –End of rotation In-Training Assessment (ITA) Report

6. V0.4 6 Competency driven training Explicitly defined competencies and role of psychiatrist Competencies are a combination of knowledge, skills and attitudes Required competencies have to be achieved to progress through the 3 Stages and to become eligible to sit the written and clinical exams

7. V0.4 7 Competency driven training What are the Roles of a Psychiatrist? What Competencies are needed for these? What are the Learning Outcomes for these Competencies? CanMEDS roles Fellowship Competencies and Developmental Descriptors Detailed Learning Outcomes for each Stage

8. V0.4 8 Competency driven training Defined syllabus for each stage of training Assessments mapped to syllabus and learning outcomes Formative assessment o formal and informal assessment procedures o involves qualitative feedback (rather than scores) Summative assessment o Tests the knowledge and understanding of information, skills, concepts or processes o Final evaluation

9. V0.4 9 What about supervision? Aspects of supervision that are different WORKPLACE-BASED ASSESSMENTS FEEDBACK AND GRADING ARE MORE FORMALISED CLEARER STANDARDS Structured workplace- based assessments Written feedback More training resources for supervisors More guidance about the standards expected in each Stage of training

10. V0.4 10 NEW - Workplace-based Assessments (WBAs) Mini Clinical Evaluation Exercise Case-based Discussion CbD Professional Presentation Observed Clinical Activity OCA Highly recommended that trainees experience these tools early in their training

11. V0.4 11 Range of WBAs You can choose: Mini-Clinical Evaluation Exercise Briefly observed clinical tasks CbD Detailed case discussions with supervisor PP Professional Presentations OCA A full 50 minutes assessment with structured feedback and clear standards At least 3 of these will help determine if an EPA is achieved

12. V0.4 12 Why Workplace-based Assessments? (WBAs) On the job assessment (efficient use of supervision time) Relevant tools to assess key clinical activities Feedback is ensured and structured Clearer standards and guidelines for the supervisor A clearer paper trail to document trainee’s performance

13. V0.4 13 WBA in the first month The first formative WBA is to be scheduled in the first month of the rotation Aim to do it within the first 4 weeks that the trainee and supervisor have worked together To help the supervisor assess the trainee’s strengths and weaknesses, and alert them to any significant problems

14. V0.4 14 STAGE 1 STAGE 2 STAGE 3 BASICPROFICIENT ADVANCED OCI SCHOLARLY PROJECT EPAs and SUPERVISOR ASSESSMENTS OSCE Summative assessments WRITTEN PSYCHOTHERAPY WRITTEN CASE

15. V0.4 15 CBFP changes CBFP training changes - any questions or discussion? Workplace-based Assessments - Professional Presentation - Observed Clinical Activity (OCA) Entrustable Professional Activities Supervisor Assessments (mid and end of rotation)

16. V0.4 16 Professional Presentation Where and when? o Journal clubs o Case presentations o Grand Round presentations o Teaching medical students (didactic presentation) o In-service presentations o Wider options may be possible for more senior trainees Presentation needs to run for 30 minutes Supervisor must attend for it to be used as a WBA

17. V0.4 17 Professional Presentation Assessment criteria o Introduction of the topic or issue o Context of information and material used o Analysis and critique o Presentation and delivery o Responses to answer questions o Quality of educational content Get trainee to self-assess as well as grading them Set time to have a feedback discussion shortly afterwards

18. V0.4 18 Professional Presentation There will be degrees of difference amongst trainees: –level of supervisor involvement with the preparation –trainee’s experience with presentations –audience for the presentation: o local colleagues o in-service to other staff o medical staff at a local hospital o conference

19. V0.4 19 Professional Presentation How to manage the grading, given that all this will vary? It’s a Formative Workplace-based Assessment You do this anyway, with your trainees This formalises Professional Presentations as a more structured and specific way to give feedback so as to shape their skills General Discussion and questions

20. V0.4 20 Observed Clinical Activity (OCA) Trainee nominates a clinical case Patient chosen will be new to the trainee OCA involves 2 sessions (1 hour each) 1st session o 50 minutes interview – trainee assessment of a real patient o 10 minutes post interview verbal feedback 2nd session o Use the 1 hour of 1:1 supervision time for case presentation, discussion and feedback

21. V0.4 21 Observed Clinical Activity (OCA) After the interview, trainee presents the case - allow more time for 1 st years Structured written feedback using the OCA marking sheet – again, get the trainee to self-assess as well, then compare notes Formative assessment to shape practice If trainee is below the standard, supervisor can: o do an additional OCA o Repeat part that was below standard, e.g. case presentation o do other WBAs focusing on specific skills

22. V0.4 22 Observed Clinical Activity (OCA) OCA mark sheet - Discussion Note the areas to mark: History taking process History taking content Mental state & relevant physical exam skills Clinical synthesis Management plan

23. V0.4 23 OCA role play 30 minutes – divide into pairs OCA feedback o Information for the ‘supervisor’ o Information for the ‘trainee’ Mark sheets o Trainee self-assessment mark sheet o Supervisor version of mark sheet Reconvene, discussion and questions

24. V0.4 24 BREAK Reconvene in 15 minutes

25. V0.4 25 Summative assessments Entrustable Professional Activities –information and practical exercise In-Training Assessments (ITAs) (Supervisor Assessments) –information and discussion of vignette

26. V0.4 26 Reminder - Tasks in Stage 1 training ROTATION 1ROTATION 2 Formative mid rotation Supervisor assessment EPA 1 Summative EPA 1 Summative Formative mid rotation Supervisor assessment WBA Formative WBA Formative Minimum of 3 WBAs for 1 EPA Supervisor assessment Summative WBA Formative WBA Formative WBA Formative WBA Formative WBA Formative EPA 2 Summative EPA 2 Summative EPA 3 Summative EPA 3 Summative EPA 4 Summative EPA 4 Summative WBA Formative WBA Formative WBA Formative WBA Formative WBA Formative Supervisor assessment Summative First 4 weeks

27. V0.4 27 Entrustable Professional Activities (EPAs) a reminder EPAs relate to activities identified as core to the profession 2 mandatory EPAs per rotation Despite signing off an activity as ‘entrusted’, the supervisor remains responsible to oversee the trainee’s clinical performance

28. V0.4 28 EPAs in Stage 1 Stage 1: 4 mandatory EPAs active contribution to the multidisciplinary team meeting producing discharge summaries and organising appropriate transfer of care initiating an antipsychotic medication in a patient with schizophrenia communicating with a family about a young adult’s major mental illness

29. V0.4 29 EPAs in Stage 2 that can be entrusted in Stage 1 Stage 2: 5 General Psychiatry EPAs Demonstrating proficiency in all the expected tasks associated with prescription, administration and monitoring of ECT The application and use of the Mental Health Act Assessment and management of RISK of harm to self and others Safe and effective use of Clozapine in psychiatry Cultural Competence

30. V0.4 30 EPAs in Stage 2 that can be entrusted in Stage 1 Stage 2: 2 of 4 Psychotherapy EPAs The provision of psychoeducation in a formal interactive session Psychodynamically informed patient encounters and managing the therapeutic alliance Visit the EPA Forms page for more information.EPA Forms

31. V0.4 31 Entrustable Professional Activities (EPAs) Not a single ‘activity’ but is based on at least 3 preceding formative WBAs plus general observation and assessment of the trainee Signed off generally in the second 3 months of a rotation Trainee does a self-assessment and gives it to the supervisor beforehand, when they feel they have achieved the requirements Feedback session about an EPA is scheduled during the weekly hour of 1:1 supervision

32. V0.4 32 Entrustable Professional Activities (EPAs) Summative assessments Trainees do not ‘fail’ an EPA. They fail to have them entrusted in the time expected. Review progress towards the required EPAs at mid rotation. If trainee can’t have them entrusted yet they’ll need to do more work, with feedback, to meet the standard. May indicate areas needing remedial work – discuss concerns with the Director/Coordinator of Training

33. V0.4 33 Entrustable Professional Activities (EPAs) Discuss the 4 EPAs of Stage 1 in detail Review the form to document achievement Discussion and questions

34. V0.4 34 In-Training Assessments (ITAs) Before Mid-rotation – formative End of Rotation – summative (Pass or Fail decision) What has changed? More detailed Learning Objectives to be discussed with trainee and graded More details available about the required standard (refer Developmental Descriptors)

35. V0.4 35 Mid-Rotation ITA Form - formative Formative feedback and discussion Ask trainee to self-assess and discuss their grades Alert DoT if there are any concerns at this point Remedial plan if there are significant concerns WBAs o formative o help determine trainee’s knowledge, skills & attitudes o use them in your mid-rotation feedback o some WBAs are required for all trainees, but extras can be added as needed

36. V0.4 36 Mid-rotation ITA Form Discuss the mid-rotation ITA form and process Developmental Descriptors are guides to the standard College website for latest versions of documents

37. V0.4 37 Mid-rotation ITA Form – vignette You are not sure how to grade the Learning Outcome: “Discuss reasonable diagnostic formulations and differential diagnoses for presenting problems, using the DSM or ICD systems” Your trainee summarises rather than formulates cases. They can usually suggest reasonable DSM differential diagnoses for straightforward cases, but they don’t make explanatory links to the patient’s history except for obvious biological causes like head injury or a strong genetic loading. However, Trainee has only done 9 months in Stage 1 so far.

38. V0.4 38 Mid-rotation ITA Form – vignette What standard do the Developmental Descriptors say they should be at? By the end of Stage 1 “Produces and accurate Biopsychosocial model formulation and requires supervision to link salient factors.” So can they? They can think up biological factors but not psycho-socio-cultural ones. Looking at the Stage 2 standard may help – next slide

39. V0.4 39 Mid-rotation ITA Form – vignette How does this Stage 1 standard compare with what’s expected by the end of Stage 2? (see the DDs handout) “Able to identify and succinctly summarise important aspects of the history, using a BPS framework, and develop hypotheses as to how these factors interacted such that the patient now presents with the problems identified. Clearly demonstrates an understanding of the individual before them (i.e. tailored and not generic formulations). Hypotheses should be based on recognised psychological, social and biological theories and, where extant, evidence. Such theories and evidence that the candidate relies upon should be accurately described and applied in a manner that demonstrates a deeper level of understanding. These hypotheses should inform management recommendations. Uses supervision to assist and learn from this process.”

40. V0.4 40 Mid-rotation ITA Form – vignette FAILPASS No Data yet Rarely Met Inconsistently Met Almost Always Met Sometimes Exceeded Consistently Exceeded Discuss the vignette What grade would you give them? How might that affect the final 3 months of Stage 1?

41. V0.4 41 End-of-Rotation ITA Report - summative Summative grading process at end of rotation With formative feedback and discussion – get trainee to self-assess and bring in their version of the grades You may also need to discuss and sign off EPAs Formative WBAs o Will again help determine trainee’s knowledge, skills & attitudes o use them in your end of rotation feedback to trainee and to determine the grading

42. V0.4 42 End-of-Rotation ITA Report The grading is more detailed Discussing it will probably take longer – maybe 2 x 1 hr supervision sessions, especially if EPAs also need to be discussed Need to think through and pencil in your grades beforehand, referencing the Developmental Descriptors as a guide as needed

43. V0.4 43 End-of-Rotation ITA Report Review the new End-of-Rotation ITA Report form Discuss the form and the process What if the trainee has not improved? How would you grade them now? What are the issues?

44. V0.4 44 End-of-Rotation ITA Report For the poorly performing end of Stage 1 trainee: Weigh up all the evidence Discuss with DoT or local Training Coordinator Be aware of the consequences of a Failing grade with the End-of-Rotation ITA or of not achieving the ST1 EPAs. Review the Progression through Training and Failure to Progress Policies.Progression through Training and Failure to Progress Policies

45. V0.4 45 Supervisors are crucial In summary – in the new CBFP model: There are more summative assessments To ensure the overall standard of trainees’ performance in meeting standards Supervisor is manager, assessor as well as mentor and teacher Contact the Local Training Coordinators and DoTs early for support and assistance

46. V0.4 46 Supervisors are crucial Consequences of lack of clear feedback to a trainee not performing adequately Trainees’ anxieties & inadequacies are carried onto the next rotation There may be exam failures later Weaknesses exposed later may be difficult to accept due to previous ‘good reports’ Reputation of prior supervisors may be devalued when trainees receive more honest and frank feedback later Possible competence and safety problems with poor clinical care of patients

47. V0.4 47 Structures and limits for supervisors Registration and recertification Contractual obligations (e.g. provision of teaching, training, clinical work, administration) College requirements (supervisor training, CPD) Maintenance of personal boundaries in the workplace – with patients, trainees, colleagues and other staff With all this, a sensible work-life balance is important

48. V0.4 48 Boundaries Close personal relationships with current trainees (Ethical Guideline 10 Fellowship Attainment Committee March 2009) –power differential –impacts negatively on learning –blurs assessor objectivity –potential for harm to trainee when relationship ends Why emphasise this? Supervisors will do more summative assessments – maintaining a professional relationship is vital

49. V0.4 49 Final Wrap-up Questions Final Discussion Certificates of Attendance