Blueprint analysis and feedback based on the Progress Test Lee Coombes Jeff Stratford

Progress Test 125 MCQ items, 4 times a year Dont Know option (negative marking) 1 formative test, 19 summative All students take the same test Set at the level of a newly qualified doctor

Progress Test

Student Feedback

Blueprinting Based on the GMC PLAB blueprint 17 domains, 17 Skills Classified by main question theme Some areas have more questions than others Aim has been to provide detailed feedback on each area for students Both single test and aggregated yearly average

Student Feedback ATLAS

Questions How can we use the information? – Provides valuable feedback Differences in gender/ethnic groups? – Weve previously seen gender and ethnicity differences in the data – Are these general, or specific to certain areas?

Methods Tests covering 7 year period – 2006/07 to 2012/13 Examining the 20 tests students take Different tests are grouped together No adjustment for test difficulty # items varies in each test/skill/domain

PCMD Progress Test

Gender

Domain: NeurologicalSkill: Pathology

Gender Domain: Women's HealthSkill: Diagnosis

Ethnicity

Domain: NeurologicalDomain: Digestive

Ethnicity Skill: Management PlanSkill: Diagnosis

Conclusions Gender differences are skill/domain specific Ethnic difference appears more general Caused by the test, or is the test accurately measuring pre-existing differences? – Evidence of similar differences elsewhere What role do other demographics play? – Home\EU\Other – Entry Type (Graduate\Adult)

Thank You Lee Coombes Jeff Stratford