1. nMRCGP The Clinical Skills Assessment : an evolving process
Prepared by the CSA Operations Group
nMRCGP

2. Health Warning
Contemporary
as of May 2007

4. Principles of Assessment (from PMETB)
The assessment system must be fit for a range of purposes
Principle 2
The content of the assessment will be based on curricula for postgraduate training which themselves are referenced to all of the areas of Good Medical Practice
Principle 3
The methods used within the programme will be selected in the light of the purpose and content of that component of the assessment framework.
Principle 4
The methods used to set standards for classification of trainee’s performance / competence must be transparent and in the public domain
PMETB = Postgraduate Medical Education and Training Board.
Oversees postgraduate medical training in all specialties. All Royal Colleges postgraduate accreditation now accountable to PMETB.

5. Principles of Assessment (from PMETB) continued
Assessments must provide relevant feedback
Principle 6
Assessors / examiners will be recruited against criteria for performing the tasks they undertake
Principle 7
There will be Lay input in the development of assessment
Principle 8
Documentation will be standardised and accessible nationally.
Principle 9
There will be resources sufficient to support assessment

6. The MRCGP Curriculum Statements (Principle 2)
Where to find them
RCGP website
What are they?
Series of statements, each covering different clinical and practice management areas, based on European Academy of Teachers in General Practice (EURACT) framework and Good Medical Practice (GMC document)
Written by a variety of experts in their field, coordinated by RCGP
Principle 2: The content of the assessment will be based on curricula for postgraduate training which themselves are referenced to all of the areas of Good Medical Practice.

7. The MRCGP Curriculum Statements (Principle 2) continued
How they are being used
Curriculum statements subdivided into ‘Intended Learning Outcomes’. Cases linked to specified learning outcomes within specified curriculum statements.
This enables sampling from across the curriculum, as cases can be mapped to the curriculum statements (or nMRCGP blueprint)

8. Components of the nMRCGP
The Applied Knowledge Test
Computer-based test of knowledge using multiple-choice questions, completed on-screen in dedicated Pearson Vue driving test centres around the country.
The Workplace Based Assessment
Formative and summative; variety of measures; based on a series of reviews; electronic portfolio. Tests trainee in his/her place of work, doing what he/she actually does.
The Clinical Skills Assessment
Clinical consulting skills examination, based on cases from general practice, with role players as ‘patients’, and experienced assessors; provides a pre-determined, standardised level of challenge to candidates.
Summative assessment and MRCGP will be replaced by an assessment package of 3 components:
WBA – take place throughout the 3 yr training
AKT – machine marked test of applied knowledge, can be sat as soon as candidate feels ready
CSA – probably at the start of the 3rd year.
Leads to Certificate of Completion of Training (CCT), which automatically makes candidate eligible for Membership of the RCGP.

9. Why a Clinical Skills Assessment?
Criticism of current MRCGP that there is no clinical consulting skills component
Provides external validation / triangulation with the other testing methods used
Using simulated patients is a validated and reliable method for testing clinical skills, so long as quality assurance of case production, role player and assessor training is carried out.
Able to offer a standardised, pre-determined level of challenge to candidates and to vary this level of challenge as needed by the assessment requirements

10. Purpose of the CSA :
‘An assessment of a doctor’s ability to integrate and apply appropriate clinical, professional, communication and practical skills in general practice’
Integrative skills assessment - tests a doctor’s abilities to gather information and apply learned understanding of disease processes and person-centred care appropriately in a standardised context, making evidence-based decisions, and communicating effectively with patients and colleagues.
Within the context of the CSA blueprint!
Look at case writing guidelines document: final 2 pages shows the ‘exploded’ discussion for each generic competency, as applied to the CSA. Helpful in writing cases as it helps to focus the case within the allocated domain (s).
The emphasis is on clinical management both with respect to routine problems and those that are less straightforward, integrated with communication, attitudinal and practical skills with which clinical care is associated.

11. Series of Integrated Developments
Curriculum statements form the basis for the nMRCGP blueprint :
Intended learning outcomes - track from curriculum statements to cases via CSA blueprint and case selection blueprint
Competency progression
CSA Blueprint – based on nMRCGP blueprint, those sections that can be tested by this methodology

12. Series of Integrated Developments continued
Case writing proforma that guides case writers through complexities of linking their case, focussing on the ‘nub’ of the case, and writing a marking schedule that reflects this ‘nub’.
Linking the case with searchable keywords so that a defined selection of cases can be found for each time the assessment is run.

13. CSA Blueprint derived from the Curriculum
Blueprint area
Descriptor
Data gathering and interpretation
Gathering of data for clinical judgement, choice of examination, investigations and their interpretations
Management
Recognition and management of common medical conditions in primary care. Demonstrates flexible and structured approach to decision making
Co-morbidity & health promotion
Demonstrating ability to deal with multiple complaints and co-morbidity and to promote positive approach to health
Person-centred approach
Use of recognised communication techniques that enhance understanding of a patient’s illness and promote a shared approach to managing problems
Professional attitude
Practising ethically with respect for equality and diversity in line with accepted codes of professional conduct
Technical skills
Demonstrating proficiency in performing physical examinations and using diagnostic and therapeutic instruments

14. Case Selection Blueprint
Clinical Skills Assessment Case Selection Blueprint
Primary nature of case
Primary system or area of disease 
Acute Illness
Chronic Illness
Undifferentiated Illness
Psychol and Social
Preventive
/lifestyle
Other
Cardiovascular
Respiratory
Neurological/ Psychiatric
Musculo-skeletal
Endocrine/ Oncological
Eye/ ENT/ Skin
Men/ Women/ Sexual Health
Renal/ urological
Gastro-intestinal
Infectious diseases
ETC
Case selection criteria beyond the agreed rubric for the matrix:
Gender mix
Diversity eg culture, disability, religion and sexual orientation
Age mix (child, older person).

15. How does the CSA differ from the Simulated Surgery?
Not just a test of communication skills in a clinical setting
Based on the nMRCGP blueprint, and samples across this blueprint.
Will be taken by many more candidates (3, ,000 per year)
Assesses integrative clinical skills in primary care settings
Candidates will be expected to undertake focussed physical examinations on some of the role players and be assessed on it

16. What is the CSA likely to look like?
Starts October 2007
Temporary assessment centre to be used initially in Croydon
Dedicated assessment centre within new College build planned to be ready over the next 3-5 years
Will take place for a number of weeks, several times a year
Will use multiple circuits
Candidate stays in ‘surgery’ and patient and examiner move around circuit
Still open to change, as we learn from experience and collect expert opinion on the methodology.
Ready to roll by:

17. What is the CSA likely to look like? continued
Will consist mostly of simulated patient cases.
13 stations, probably each of 10 minutes
Marks collected by Opscan techniques
Some triangulation with Workplace Based Assessment competencies
Stations picked from intended learning outcomes (ILOs) across the nMRCGP blueprint with clear derivation

19. The Marking Schedule Each case is marked in 3 domains :
Data gathering, examination and clinical assessment skills
Clinical management skills
Interpersonal skills
All domains have equal weighting

20. The Marking Schedule continued
Assessor uses word pictures to help decide grade for each domain, then uses this information to make a judgement on the grade for the case overall (4 decisions)
Feedback to candidates
Serious concerns box

21. Four possible Grades Clear Pass (CP) Marginal Pass (MP)
Marginal Fail (MF)
Clear Fail (CF)

22. Three domains for each case :
DATA-GATHERING, TECHNICAL & ASSESSMENT SKILLS :
Gathering & using data for clinical judgement, choice of examination, investigations & their interpretation. Demonstrating proficiency in performing physical examinations & using diagnostic and therapeutic instruments. (Blueprint: Problem-solving skills, Technical Skills)
CLINICAL MANAGEMENT SKILLS :
Recognition & management of common medical conditions in primary care. Demonstrating a structured & flexible approach to decision-making. Demonstrating the ability to deal with multiple complaints and co-morbidity. Demonstrating the ability to promote a positive approach to health. (Blueprint: Primary Care Management, Comprehensive approach)
INTERPERSONAL SKILLS:
Demonstrating the use of recognised communication techniques to understand the patient’s illness experience and develop a shared approach to managing problems. Practising ethically with respect for equality and diversity, in line with the accepted codes of professional conduct. (Blueprint: Person-centred approach, Attitudinal aspects).

23. How the CSA is aiming to meet PMETB assessment criteria
1. This methodology is judged to be the best way to test Clinical Skills in general practice currently.
2. Cases are based on the nMRCGP curriculum.
3. Assessment methodology chosen is fit for purpose. This has been validated and reliable, both elsewhere and by our main pilot.
4. Standard setting will be transparent and in the public domain with wide consultation.
5. Feedback will be given to all candidates.

24. How the CSA is aiming to meet PMETB assessment criteria : continued
6. Recruitment of assessors will be on ability – to rank order, mark reliably, knowledge base.
7. Lay input has been consistently sought.
8. Documentation will be accessible nationally through the College website and publication in peer reviewed journals and the various weekly publications.
9. Resources? Continually under review…

25. Further resources CSA sample cases to be made available soon
Latest information on RCGP website
Introduction to CSA cases document

26. Acknowledgements This presentation written by Kamila Hawthorne
Neil Munro
On behalf of the CSA Operations group, nMRCGP
Updated May 2007

27. Members of the CSA Operations Group
Adrian Freeman GP, FRCGP Examiner, Chair of the Operations Group
Mei Ling Denney GP, MRCGP Examiner and Course Organiser, Deputy Convenor MRCGP Simulated Surgery
Kamila Hawthorne GP and Trainer, MRCGP Examiner, Senior Lecturer in General Practice, Cardiff.
Fiona Patterson Professor of Work Psychology at City University
Sue Rendel GP, MRCGP Examiner and Convenor MRCGP Simulated Surgery
Amar Rughani GP, MRCGP Examiner, nMRCGP blueprint guardian
David Sales GP, Assessment Fellow, RCGP
Richard Wakeford MA Cpsychol, FRCGP, Assessment Consultant, University of Cambridge.
Faye Geoghegan nMRCGP Project Manager