The Big Picture – curricula, the Gold Guide and the assessment system Training the Trainer/Assessor Workplace-based assessment East Midlands Deanery Wednesday 28 January 2009 The Big Picture – curricula, the Gold Guide and the assessment system Joanne Brinklow joanne.brinklow@rcpath.org Head of Educational Standards

Aims Postgraduate Medical Education and Training Board (PMETB) Curricula Assessment systems Quality Framework Gold Guide Annual Review of Competence Progression (ARCP)

PMETB standards (published July 2008)

Generic standards for training Standards for curricula and assessment systems Standards for deaneries Available on PMETB website: http://www.pmetb.org.uk

Standards for curricula and assessment systems 17 standards to meet by 2010 (15 September 2009 submission date!) Met standards 2, 4 and 11 so far Most remaining standards at least partially met Annual submission of minor amendments to curricula and assessment systems

Examples of the standards The purpose of the curriculum must be stated, including linkages to previous and subsequent stages of the trainees’ training and education (1) The overall purpose of the assessment system must be documented and in the public domain (2) The choice of assessment method(s) should be appropriate to the content and purpose of that element of the curriculum (8)

Curricula: Chemical Pathology: http://www.rcpath.org/resources/pdf/g043_chempathcurriculum_aug08.pdf Histopathology: http://www.rcpath.org/resources/pdf/g051_histocurriculum_aug08.pdf Medical Microbiology and Virology: http://www.rcpath.org/resources/pdf/g042_medmicvircurriculum_aug08.pdf

College assessment system approved by PMETB Workplace-based assessment (including MSF) Year 1 assessment FRCPath Part 1 (test of knowledge) FRCPath Part 2 (test of skills and applied knowledge) Feedback on regular basis (WBA) Educational supervisor’s report ARCP (Annual Review of Competency Progression)

The College assessment system: Miller’s pyramid WBA – MSF, CbD Does Action WBA e.g. Mini-CEX, DOPS Shows FRCPath Part 2 Practical Performance Knows how Competence Knows FRCPath Part 1 Knowledge

Value of multiple assessments Key is several modes of assessment – triangulation A single exam system (FRCPath) does not provide adequate coverage of clinical practice WBA is also not sufficient in itself

Blueprints Assessments must systematically sample the entire content, appropriate to the stage of training, with reference to the common and important clinical problems that the trainee will encounter in the workplace and to the wider base of knowledge, skills and attitudes demonstrated through behaviours that doctors require (Standard 4)

PMETB’s Quality Framework The purpose of the Quality Framework is to: Provide public and professional reassurance about the standards and quality of postgraduate medical education in the UK, through a robust, rigorous set of processes; Reflect fully the principles of good regulation, demonstrate value for money and be fit for purpose;

Enable improvement and enhancement of the quality of postgraduate medical education; and Ensure specialty focus is maintained at local and national level by working with the Academy of Medical Royal Colleges (AoMRC), colleges/faculties and postgraduate deaneries. http://www.pmetb.org.uk/index.php?id=qualityframework

QA/QM/QC of postgraduate medical education and training Local education providers Quality Management Quality Assurance Quality Control Postgraduate deaneries PMETB Colleges/faculties/ associations QA/QM/QC of postgraduate medical education and training

SHARED EVIDENCE includes minimum data set PMETB QF RESPONSES TO CONCERNS VISITS to Deaneries SURVEYS STANDARDS Includes approval and action planning SHARED EVIDENCE includes minimum data set

Guide to Specialist Registrar Training The Orange Book 1998

The Gold Guide Replaced the ‘Orange Guide’ for training First published June 2007 Re-published June 2008 Available on MMC website: http://www.mmc.nhs.uk Sets out ARCP process

What is the ARCP? The Annual Review of Competence Progression is “a formal process which uses the evidence gathered by the trainee, relating to his/her progress in the training programme” (Gold Guide, page 64) Replaces RITA for StRs

College input to ARCP Pathology specific guidance and forms and specialty specific educational supervisors structured report Amended ARCP form to incorporate information about End of Stage Review All available on College website http://www.rcpath.org/index.asp?PageID=1546 To be facilitated in due course by new web-based system

Annual Review of Competence Progression Educational Appraisal Annual Review of Competence Progression Outcome Annual Planning 1. Learning agreement: aims & intended learning outcomes based on specialty curriculum 2. Advice on portfolio 3. Regular feedback (2 way) 4. Personal Development Plan 5. Trainer’s structured report 6. Workplace based (NHS) appraisal Educational supervisor and/or TPD meet with trainee to review competence outcome with trainee plan next part of training Based on a paper from PMETB’s Workplace Assessment Group (2005) and Gold Guide Workplace based (NHS) appraisal ARCP A. Evidence 1. Assessment of performance e.g. workplace based assessments and observational methods E.g. mini-CEX, CBD, DOPS, ECE examinations structured report 2. Assessment of experience, e.g. portfolio/log book audit research critical incidents B. Annual Competence Review Appropriately constituted panel considers evidence outcome Review of evidence of progress Outcome of review Educational Planning

Features of the ARCP Deanery-led process “The trainee should not normally attend the panel…unless there may be an unsatisfactory outcome” (Gold Guide, pg. 66/67) Additional or remedial training following an ARCP outcome limited to one year (exceptionally two) within the total training programme duration

Educational Supervisors structured report Role of the educational supervisor defined in the curricula Must complete a structured report for their trainees’ ARCP The structured report summarises the trainee’s portfolio Trainee’s responsibility to provide the documentation to the supervisor

What should the report include? The report should include: a summary of the WPBA outcomes outcomes from the MSF (if relevant) and Year 1 Assessment Logbook/portfolio summary of audit, research, publications, teaching, management, presentations, courses etc.

ARCP outcomes (9 in total) 1. Achieving progress 2. Development of specific competences required 3. Inadequate progress 4. Released from training programme 5. Incomplete evidence 6. Recommended for completion

End of Stage Review (ESR) The College monitors the trainee’s progress through both the year and stage of training. ARCP outcome panel must consider whether the trainee has completed the requirements for the stage of training and record this on the form as appropriate. If the ESR is not in phase with the ARCP, satisfactory completion of the stage of training can be agreed between ARCPs but must be documented.

Observations from the RCPath (so far) Only StRs should undertake ARCPs All information on the outcome form must be completed in full Only tick one outcome box Ensure that the trainee, deanery and College all receive a copy of the outcome form Use/refer to College guidance and forms

The Big Picture – curricula, the Gold Guide and the assessment system Training the Trainer/Assessor Workplace-based assessment East Midlands Deanery Wednesday 28 January 2009 The Big Picture – curricula, the Gold Guide and the assessment system Joanne Brinklow joanne.brinklow@rcpath.org Head of Educational Standards