1. Blueprint analysis and feedback based on the Progress Test Lee Coombes Jeff Stratford

2. Progress Test 125 MCQ items, 4 times a year Dont Know option (negative marking) 1 formative test, 19 summative All students take the same test Set at the level of a newly qualified doctor

3. Progress Test

4. Student Feedback

5. Blueprinting Based on the GMC PLAB blueprint 17 domains, 17 Skills Classified by main question theme Some areas have more questions than others Aim has been to provide detailed feedback on each area for students Both single test and aggregated yearly average

6. Student Feedback ATLAS

7. Questions How can we use the information? – Provides valuable feedback Differences in gender/ethnic groups? – Weve previously seen gender and ethnicity differences in the data – Are these general, or specific to certain areas?

8. Methods Tests covering 7 year period – 2006/07 to 2012/13 Examining the 20 tests students take Different tests are grouped together No adjustment for test difficulty # items varies in each test/skill/domain

9. PCMD Progress Test

10. Gender

11. Domain: NeurologicalSkill: Pathology

12. Gender Domain: Women's HealthSkill: Diagnosis

13. Ethnicity

14. Domain: NeurologicalDomain: Digestive

15. Ethnicity Skill: Management PlanSkill: Diagnosis

16. Conclusions Gender differences are skill/domain specific Ethnic difference appears more general Caused by the test, or is the test accurately measuring pre-existing differences? – Evidence of similar differences elsewhere What role do other demographics play? – Home\EU\Other – Entry Type (Graduate\Adult)

17. Thank You Lee Coombes Jeff Stratford