Presentation is loading. Please wait.

Presentation is loading. Please wait.

Implementing Work-Based Assessments Professor T.Masud Nottingham University Hospitals NHS Trust.

Similar presentations

Presentation on theme: "Implementing Work-Based Assessments Professor T.Masud Nottingham University Hospitals NHS Trust."— Presentation transcript:

1 Implementing Work-Based Assessments Professor T.Masud Nottingham University Hospitals NHS Trust

2 ST1 ST2 Core Training GIM (Acute) L1 Curriculum F1 F2 FP Curriculum ST3 ST4 ST5 ST6 ST7 Speciality Curriculum GIM (Acute) L2 Curriculum Generic C. L1 Generic Curriculum L2 CCT in Geriatric Medicine JRCPTB Cert. in GIM L2 SelectionAllocation MRCP Parts 1, 2 (written), PACES KBA ( specialist exam) WORK BASED ASSESSMENTS Overview- Curricula and Assessments for Training *Tooke report recommends separating F1 and F2 & joining F2 to ST1+2

3 Generic Curriculum - builds on Foundation Curriculum (competencies categorised into Knowledge, Skills, Attitudes & Behaviour) Level 1 (Mandatory CT Competencies) Level 1 (Mandatory CT Competencies) –1.1 History taking, examination, record keeping –1.2 Time management and Decision Making skills –1.3 Good Quality Care and Patient Safety –1.4 Infection Control –1.6 Valid Consent –3.1 Communication wih patients within a consultation –Focus Area 4 Working with Colleagues –Focus Area 6 Professional Behaviour

4 Generic Curriculum - builds on Foundation Curriculum (competencies categorised into Knowledge, Skills, Attitudes & Behaviour) Level 2 Competency Areas Level 2 Competency Areas –1.5 Health Promotion and Public Health –1.6i Medical Ethics and Public Health –1.6iii Legal framework for Practice –1.7 Ethical research –1.8 Managing Long Term Conditions and Promoting Patient Self-Care –Focus Area 2 Governance + Maintaining Good Clinical Practice –3.2 Breaking Bad News –3.3 Complaints and Medical Error –Focus Area 5 Teaching and Training –Focus Area 7 Management and NHS Structure

5 Curriculum for GIM (Acute Medicine)– Levels1+2 (Categorised into levels 1 & 2 by Knowledge, Skills, Attitudes+Behaviour) Emergency Presentations –Cardio-respiratory arrest –Shocked patient –Unconscious patient –Anaphylaxis Abdominal Pain Acute back pain Blackout/Collapse Breathlessness Chest pain Confusion- acute Cough Diarrhoea Falls Fever- Fits/seizures Haematemesis/Melaena Headache Jaundice Limb pain/swelling Palpitations Poisoning Rash Vomiting /nausea Weakness/Paralysis Top 20 Common Medical Presentations

6 Curriculum for GIM (Acute Medicine)– Levels1+2 (Categorised into levels 1 & 2 by Knowledge, Skills, Attitudes+Behaviour) Abdo mass/hepatosplenomegaly Abdo swelling/constipation Abnormal sensation Aggressive/disturbed behaviour Alcohol/substance dependence Anxiety / panic behaviour Bruising Chance findings Dialysis Dyspepsia Dysuria Genital discharge & ulceration Haematuria Haemoptysis Head Injury Hoarseness and stridor Hypothermia Immobility Involuntary movements Joint swelling Lymphadenopathy Other important presentations SYSTEM SPECIFIC COMPETENCIES Loin pain Medical complics of acute illness/surgery Medical problems in pregnancy Memory loss Micturition (difficult) Neck pain Non-organic physical symptoms Polydipsia Polyuria Pruritis Rectal bleeding Skin and mouth ulcers Speech disturbances Suicidal ideation Swallowing difficulties Syncope and presyncope Unsteadiness /balance problems Visual disturbance Weight loss Allergy Cancer & palliative care Cardiovascular medicine Clinical pharmacology Dermatology Diabetes and endocrinology Gastroenterology / hepatology Haematology Immunology Infectious diseases Medicine in the Elderly Musculoskeletal system Neurology Psychiatry Public Health / H.Promotion Renal medicine Respiratory medicine Investigation CompetenciesProcedural Competencies

7 Geriatric Medicine ST Curriculum (Jan 2007) 1 Primary learning objectives (achieve competencies in:) [various settings] 1. Perform Comprehensive Geriatric Assessment 2. Diagnose and manage acute illness in old age 3. Diagnose and manage those with chronic disease and disability 4. Provide rehabilitation with the multidisciplinary team to an older patient 5. Plan the transfer of care of frail older patients from hospital 6. Assess a patient’s suitability for and provide appropriate care to those in long term (continuing) care in the NHS or community 7. Apply knowledge and skills of a competent geriatrician in an intermediate care and /or community setting

8 Geriatric Medicine ST Curriculum (Jan 2007) 2 8. Assess and manage older patients presenting with the common geriatric problems (syndromes): a. Falls- with and wihout fracture a. Falls- with and wihout fracture b. Delirium b. Delirium c. Incontinence c. Incontinence d. Poor mobility d. Poor mobility 9. Demonstrate competence in following subspecialities: a. Palliative care a. Palliative care b. Orthogeriatrics b. Orthogeriatrics c. Old Age Psychiatry c. Old Age Psychiatry d. Specialist Stroke Care d. Specialist Stroke Care 10. Be competent in a. Research methodology (basic) a. Research methodology (basic) b. Ethical principles of research b. Ethical principles of research c. Critical appraisal of medical literature c. Critical appraisal of medical literature d. Preferably to have personal experience of research d. Preferably to have personal experience of research [basic science or clinical (health service)] [basic science or clinical (health service)]

9 Assessment Plan in Geriatric Medicine - Overview MRCP PACES KBA (SE) Work Based Assessments –Mini-CEX, DOPS, ACAT, CbD, MSF, PS Assessment of Generic Areas –Research (portfolio, supervisor reports, publications) –Audit (portfolio, audit reports, supervisors reports) –Clinical Governance (portfolio, supervisor reports) –Teaching (portfolio, assessed teaching) Regular Appraisal by Educational Supervisor (including Educational Supervisor Reports) Training Record –educational supervisor + consulatant trainer reports –completed + signed work based assessments –reflective learning –course certificates, audit reports, publications etc

10 Work Based Assessment Methods Mini-Clinical Evaluation Exercise (mini-CEX) Direct Observation of Procedural Skills (DOPS) Multi-Source Feedback (MSF) Case-Based Discussions (CbD) Patient Survey (PS) Acute Care Assessment Tool (ACAT)

11 Mini-CEX (ST3-ST7) [n=25] 4 Acute (1 ward round) 2 Rehab (1 ward round) 1 Pre-op Orthogeriatric 1 Post-op Orthogeriatric 2 Chronic disease (clinic eg DM, OA) 1 MDT chair (discharge) 1 Continence 1 Falls 1 Movement disorder 1 Delirium / Depression 1 Old Age Psychiatry (HV / Ward referral) 1 Osteoporosis / metabolic bone disease 1 Comprehensive Geriatric Assessment 1 Intermediate care / Home visit 1 Continuing care 1 Day Hospital 3 Stroke (acute WR, Rehab WR, TIA clinic 1 Palliative care (break bad news)

12 CbD (ST3-ST7) [n=26] 4 Acute (diagnosis, Mx, prescribing) 2 Rehab 2 Continuing care (≥ 1 non-NHS) 1 Evidence Based Medicine 1 Ethics / Law 1 Health Promotion 1 Complaint 1 Intermediate Care 1 Transfer of Care problem 1 Delirium 1 Old Age Psychiatry 1 Depression / dementia 1 Falls 1 Continence 1 Orthogeriatric – acute 1 Orthogeriatric - rehab 1 Acute Stroke 1 Rehab Stroke 1 Neurovascular investigation (TIA) 1 Palliative care 1 Tissue Viability / Hypothermia

13 Mid-Trent Experience of MSF (360 degree appraisals for SPRS)

14 Overview of Earlier Pilot Early pilot June-August 2004 18 SPRs in 6 NHS Trusts Then 13 SPRs (1 abroad, 1 maternity leave) 11 SPRs performed 360 degree appraisals Minimal guidance Experience discussed at RITAs Sept 2004 (subjective) Questionnaire sent to SPRs in Jan 2005 (objective)

15 Who decided which people the form should be sent to? No. Ward Manager

16 Did you find the exercise useful? Yes- extremely useful 1 Yes- quite useful 4 Not sure Not sure 3 No- not that useful 2 No- waste of time 1

17 Comments: Those who found process useful Gave useful feedback on how I am performing on the ward I think it is a great way of assessing clinicians Getting positive feedback improved my confidence. The process informed the appraisal meeting with my educational supervisor

18 Comments: Those who found process not useful Other people should distribute the forms so the SPR does not know who has been asked to complete them (x2) Meetings with the educational supervisor more important than the 360 degree appraisal

19 Further analysis Of the 5 SPRs who found the process useful, for 3 of them “others” decided who to send the forms to. Of the 6 SPRS who who did not find the process useful or were not sure, all 6 SPRs chose the people to send the forms to

20 Conclusion of the Early pilot Main hypothesis generated: The process is more useful if educational supervisor / consultant trainer decides to whom the questionnaires are sent to.

21 Randomisation- by Trust Randomised to SPRs decides (n=8) ES/Con decides (n=7) 3 Trusts Questionnaire sent to SPRs and ESs after the RITAs in Oct 2005

22 Mean no of Qs sent 17.8 (range 10-20) Mean returns 15.1 (range 10-20) Mean response rate 85% (range 55%- 100%) No difference between the 2 groups

23 Did you find the process useful? Total SPR Decides ES/Con Decides Yes- extremely 202 Yes- quite 945 Not sure 220 No- not that useful 220 No- waste of time 000 “50% useful” “100% useful” [Chi sq 4.77, df 1, p=0.029; Fishers p=0.051]

24 Mini-CEX 12 SPRs had performed at least 1 Mini-CEX 9 found them useful, 3 not Egs - Ward round (x4); MDM (x2); Tilt (x1) - Rest not stated

25 DOPS + Patient Satisfaction Qs 4 SPRs had performed at least 1 DOPS 3 found them useful (Temp pacing, OGD, not stated) 1 not useful (LP) 1 SPR performed a patient satisfaction Q- useful

26 Feedback from Educational supervisors (5 returns from 7) 2 found the 360 degree process extremely useful 2 found it quite useful 1 not sure

27 Feedback from Educational supervisors Was the process time-consuming? No- 4 Yes–slightly 1 Yes – very 0

28 Assessment Plan For Speciality Training in Geriatric Medicine MRCP KBE (SE) Mini- CEX CbDMSFACATPS ES+CT Reports Audit / Research Teach Asses Cert ST3 * 6 Acute 6 Acute 1 *ALS ST4 6 6 * 1 * * Acute Audit ALS Res. Meth. ST5 * 6 6 1 * Audit (eg rehab) * ALS Teach ST6 6 6 * * * PYA * PYA Audit (eg IC) Res. present. ALS ST7 1 2 1 *Publication ALS Manag. TOT 25 25 26 26 2 42 Summative Formative

29 Educational Appraisal Annual Review of Competence Progression Outcome Annual Planning 1. Learning agreement:  aims & intended learning outcomes  based on specialty curriculum 2. Advice on portfolio 3. Regular feedback (2 way) 4. Personal Development Plan 5. Trainer’s structured report 6. Workplace based (NHS) appraisal Educational supervisor and/or TPD meet with trainee to  review competence outcome with trainee  plan next part of training Based on a paper from PMETB’s Workplace Assessment Group (2005 ) Workplace based (NHS) appraisal ARCP A. Evidence 1 Assessment of performance e.g.  workplace based assessments and observational methods E.g. mini-CEX, DOPS, video, CBD  examinations  structured report 2. Assessment of experience, e.g.  portfolio/log book  audit  research  critical incidents B. Annual Competence Review  Appropriately constituted panel considers evidence Outcome of review Role of the ARCP

30 Challenges Overlap between SpRs and StRs StRs – proactive Educational supervisors – proactive Consultant Trainers – proactive New documentation – structured Time in Job plans Useful process or tick-box exercise?

31 Some Suggestions StRs and CTs to have forms for CbDs, Mini-CEX handy 2 CbDs and 2 Mini-CEX per 4 month attachment 3 CbDs and 3 Mini-CEX per 6 month attachment Importance of Meeting with Education Supervisor about 1 month before ARCP Regular half day / full day teaching sessions mapped to curricula

32 Partnership StR Educational Supervisor TPDs Consultant Trainer Specialist Training Committee Deanery BGS ETC SAC JRCPTB PMETB NHS Trust Regional Advisor

33 Assessments! More Work Based Assessments

Download ppt "Implementing Work-Based Assessments Professor T.Masud Nottingham University Hospitals NHS Trust."

Similar presentations

Ads by Google