1. ‘Do we need exams?’ Wendy Reid Medical Director HEE Past – Vice President RCOG

2. Assessment of doctors.... Demanded by the public Required by the regulator Necessary for the definition of ‘profession’ But..... Opaque methods No direct input from the public Examinations are often ‘historical’ not designed for their present purpose

3. What is Assessment ? A biopsy of knowledge and skills “clinical competence”

4. Critical questions in assessment WHY are you doing the assessment? WHAT are you going to assess? HOW are you going to assess it? HOW WELL is the assessment working?

5. WHY are you doing the assessment? Is its purpose: –F–Formative? –S–Summative? Graduation/ PG Certification In course/ in training feedback

6. Critical questions WHY are you doing the assessment? WHAT are you going to assess? HOW are you going to assess it? HOW WELL is the assessment working?

7. WHAT are we testing? Clinical competence Knowledge – factual – applied: clinical reasoning Skills – communication – clinical Attitudes – professional behaviour

8. A model of clinical competence Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S67. Knows Shows how Knows how Does Professional authenticity Cognition = knowledge Behaviour = Skills + attitude

9. WHY are you doing the assessment? WHAT are you going to assess? HOW are you going to assess it? HOW WELL is the assessment working?

10. Testing formats Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S67. Knows Shows how Knows how Does Professional authenticity Written/ Computer based assessment Performance/ hands on assessment

11. Testing formats Knows Shows how Knows how Does Knows Factual tests: SBA, SAQ, (EMQ) Knows how (Clinical) Context based tests: SBA, SAQ, (EMQ) Shows how Performance assessment in Vitro: OSCE Does Performance assessment in vivo: WBA eg mini-CEX, DOPs, TBA

12. Critical questions WHY are you doing the assessment? WHAT are you going to assess? HOW are you going to assess it? HOW WELL is the assessment working?

13. How well is the assessment working? Is it valid? Is it reliable? Is it doing what it is supposed to be doing? To answer these questions, we have to consider the characteristics of assessment instruments ** Define the purpose of the assessment

14. Characteristics of assessment instruments Validity (V) Reliability (R) Educational impact (E) Acceptability (A) Cost (C)

15. Specialty Training & Education Programme 5 67 Specialist Training Curriculum CCT 34 Full registration Log Book Subspecialty 2-3yr 1 2 Advanced Training Modules Women's Health Module 1 * Foundation Annual Review of Competence (ARCP) 2 Basic Intermediate Part 1 MRCOG Exam Part 2MRCOG Exam

16. Curriculum ‘Run-through’ i.e. Appointed once, progress by assessment Iterative 7 years – average doctor takes 9.8 years First 2 years – basic knowledge, must pass part 1 of exam Middle 3 years – intermediate, must pass part 2 of exam Final 2 years – advanced, continue with core work and learning but add specialist modules

17. Principles of curriculum Competency based Performance measured Iterative time – ‘weigh’ points Transition clearly defined at each stage Flexibility in advanced training Generic skills across core Log book – e-portfolio Knowledge and application of knowledge tests Workplace based assessments

18. Aim of curriculum Produce well trained Obstetricians & Gynaecologists ready for consultant posts in the NHS Produce doctors with flexibility of career choice, well advised throughout training Produce doctors who will advance the care of women Re-defined in ‘Tomorrow’s Specialist’ publication 2012

19. Options during training Doctors are allowed to: Work less than full time (50% or more) Take time out of the programme to work overseas or do research (maximum 3 years) Can move into formal Academic training pathway ‘Pause’ – personal reasons, Olympics, Maternity leave Apply for sub-specialty training from end of year 5 But... Every doctor does the MRCOG examination

20. MRCOG Examination Any graduate can enter from anywhere in the world, need evidence of medical degree Part 1 – test of basic knowledge applied to clinical O&G. Written papers (EMQs, MCQs) Part 2- application of knowledge, 2 written elements require pass before OSCE element Reviewed in 2013 – new proposal to split part 2 and have oral element as part 3

21. The MRCOG Overseas Centres

22. Part 1 Success Rates

23. Part 2 Success Rates

24. Why Take the MRCOG? “It is one of the most highly recognised and well-respected degrees in my country” [India] “It is a window through which I can have more knowledge and find the chance of training in O&G” [Sudan] “It would give me the best chance at getting first-world training which I could use to advance the level and quality of health care service provided in my coutry” [Trinidad]

25. Why Take the MRCOG? “I wish to have an international degree with expertise in evidence-based medicine, audits and protocols…to serve patients better” [India] “Passing…means that I have achieved an appropriate level to implement RCOG standards to improve women’s health.” [Saudi Arabia] “It is a prestigious and well-recognised qualification.” [Pakistan]

26. Consultant Country of Qualification © Royal College of Obstetricians and Gynaecologists

27. Principles of Assessment

28. Yes, we need exams Public confidence Professional recognition RCOG standard International credibility – for the college nad for individuals But they must be fit for purpose, modern, reflect best educational practice and embrace evidence based techniques