1. CLICK TO GO BACK TO KIOSK MENU Correlation Coefficient
USMLE Scores Do Not Predict Ultimate Clinical Performance in an EM Residency Program
CLICK TO GO BACK TO KIOSK MENU
Karima R Sajadi*, Mark Saks#, Edward A Ramoska*
*Drexel University College of Medicine, #Crozer Chester Medical Center
Introduction
Discussion
Results
“High-stakes” multiple choice exams are widely used to gauge mastery of basic and clinical science knowledge
Scores on these exams are important screening and applicant ranking criteria
We attempt to clarify the relationship between performance on two USMLE exams (Step 1, Step 2CK) with global clinical performance in an EM residency program
We tested the hypothesis that USMLE scores do not predict clinical performance during residency training
USMLE scores are important as resident selection criteria
USMLE scores have been shown to correlate with future exams (ITE1,3, ABEM qualifying exam2,3)
However, they should not be used to predict the applicant’s actual clinical performance as judged by residency leadership
During the 8 years there were a total of 115 graduating residents
73 men (63%) and 42 women
109 (95%) had allopathic medical degrees; the remainder had osteopathic degrees
Table 1 shows the distribution of the final consensus ranking. The inter-rater reliability was strong with an intraclass correlation coefficient = (p<0.01)
There was a statistically significant correlation between our ranking of clinical performance and Step 2 CK score (Spearman’s rho)
There was no statistically significant correlation between clinical performance and Step 1 score
Conclusion
Neither USMLE Step 1 nor Step 2CK were good predictors of clinical performance during training
USMLE scores may be overemphasized in the resident selection process
Methods
All graduating residents from our PGY 1-3 University-based EM residency program with complete USMLE scores
From 2008 to 2015
Clinical performance was defined as a gestalt of the residency program’s leadership (PD/2 APDs)
Initially blinded to each other’s grouping selections
Classified the residents into 3 sets: top, average, and lowest clinical performer
Dissimilarities of the rankings were adjudicated during a consensus conference
Category
Number
Percentage
Top 38
33%
Middle 44
38%
Bottom 33
29%
USMLE Step 1
USMLE Step 2CK
Correlation Coefficient
0.067
0.205
P Value
0.49
0.04 N
109
106
References
Table 1. FINAL RANKING OF RESIDENTS
Thundiyil JG, Modica RF, Silvestri S, Papa L: Do United States Medical Licensing Examination (USMLE) scores predict in- training test performance for emergency medicine residents? J Emerg Med. 2010; 38:65-9.
Harmouche E, Goyal N, Pinawin A, Nagarwala J, Bhat R: WesterUSMLE Scores Predict Success in ABEM Initial Certification: A Multicenter Study. West J Emerg Med 2017;18:
Nelson M, CalandrellaC: Does USMLE Step 1 & 2 Scores Predict Success on ITE and ABEM Qualifying Exam: A Review of an Emergency Medicine Residency Program from its Inception Ann Emerg Med 2017: 70: S58-S59.
Table 2. CORRELATION BETWEEN CLINICAL PERFORMANCE AND EXAM SCORES