Presentation is loading. Please wait.

Presentation is loading. Please wait.

Steven Kane, MD David Robinson, MD Atlanta Medical Center.

Similar presentations

Presentation on theme: "Steven Kane, MD David Robinson, MD Atlanta Medical Center."— Presentation transcript:

1 Steven Kane, MD David Robinson, MD Atlanta Medical Center

2  Core Competencies  Prior USMLEs  Interviews  Faculty Evaluations  Intra-operative observation  Clinical observation  Surgical Skills lab  Peer to Peer  Pre medical experience  Ancillary staff evaluations  OITE  Milestones

3 9.63


5  “I shall not today attempt further to define the kinds of material I understand to be embraced within that shorthand description and perhaps I could never succeed in intelligibly doing so. But I know it when I see it……” 1964 Potter Stewart

6  Carmichael, Kelly D. MD; Westmoreland, James B. MD; Thomas, John A. MD; Patterson, Rita M. PhD  May 2005, Southern Medical Journal  The only statistically significant correlations to OITE scores were USMLE step I performance (220) and marital status (married)

7  Paul J. Dougherty MD; Norman Walter MD; Peter Schilling MD; Soheil Najibi MD, PhD; and Harry Herkowitz MD  March 2010, Clinical Orthopaedics and Related Research  Those who averaged in the 27th percentile or lower on the OITE had a 57% chance of failing the ABOS Part I examination  Correlation coefficient of 0.53 between the USMLE Step 1 and ABOS Part I  Average correlation coefficient of 0.50 for postgraduate years (PGY) 2 through 5 OITE scores and ABOS Part I

8  KLEIN, GREGG R. MD; AUSTIN, MATTHEW S. MD; RANDOLPH, SUSAN; SHARKEY, PETER F. MD; HILIBRAND, ALAN S. MD  May 2004, The Journal of Bone and Joint Surgery-American Volume  Correlation coefficient for the USMLE Step-I score and the ABOS Part-I percentile score was 0.38 (p = 0.002)  Correlation coefficients were also calculated for the OITE percentile scores for years 3 (r = 0.52, p < 0.0001), 4 (r = 0.49, p < 0.0001), and 5 (r = 0.69, p < 0.0001).  There was a high risk of failure (63%; five of eight failed) on the ABOS Part-I examination when a resident scored below the twenty-ninth percentile for PGY-3 and below the twentieth percentile for PGY-5

9  5 programs evaluated 176 current and former residents evaluated  Chairs or PD’s asked to rate overall competence and surgical competence along with the 6 core competencies while “modestly blinded” to USMLE, OITE, and ABOS percentiles


11 A statistical cross comparison of USMLE 1, USMLE 2, OITE 1-5 percentiles, 6 Core Competencies, ABOS percentiles, Surgical Competency and Overall Competency upon graduation was compiled.


13 Variableby VariableN Spearman ρ Prob> │ρ│ ABOS Part 1 PercentileOITE 1250.1700.0417 ABOS Part 1 PercentileOITE 2650.478<.0001 ABOS Part 1 PercentileOITE 3650.2960.017 ABOS Part 1 PercentileOITE 4620.2510.049 ABOS Part 1 PercentileOITE 5600.463<.0001

14 Correlation with ABOS Survey CategoryNSpearman ρ Prob> │ρ│ Patient Care650.180.1501 Medical Knowledge650.54<.0001 Practice-Based Learning and Improvement650.280.0261 Interpersonal and Communication Skills650.270.0300 Professionalism640.280.0266 Systems-Based Practice650.270.0283 Surgical Competency650.370.0026 Overall Competency650.360.0035

15 Survey/Score CategoryScore CategoryRSp - Value Medical KnowledgeABOS %tile0.548650.000002 OITE %tile YIT-5ABOS %tile0.451600.000303 OITE %tile YIT-2ABOS %tile0.423650.000446 Surgical CompetencyABOS %tile0.390650.001321 Medical KnowledgeOITE %tile YIT-20.3851560.000001 Overall CompetencyABOS %tile0.370650.002415 Medical KnowledgeOITE %tile YIT-30.3631540.000004 Medical KnowledgeOITE %tile YIT-50.3371170.000199 OTIE %tile YIT-3ABOS %tile0.317650.010091 USMLE Step 2ABOS %tile0.299230.166287 ABOS was more correlative with surgical and overall competency than OITE (medium correlation)

16 Survey/Score CategoryScore CategoryRSp - Value PBL & ImprovementABOS %tile0.298650.015810 Systems-Based PracticeABOS %tile0.290650.019020 ProfessionalismUSMLE Step 20.298530.035895 Interper. & Comm. SkillsUSMLE Step 20.288530.036435 Surgical CompetencyOITE %tile YIT-20.2811560.000382 Interper. & Comm. SkillsABOS %tile0.272650.028624 Medical KnowledgeOITE %tile YIT-40.2601370.002153 Surgical CompetencyOITE %tile YIT-30.252620.048205 OITE %tile YIT-4ABOS %tile0.252620.048205 Overall CompetencyOITE %tile YIT-30.2341540.003492 OITE YIT- 4 Appears to be a throw away year

17 Survey/Score CategoryScore CategoryRSp - Value Surgical CompetencyOITE %tile YIT-10.155890.1464 Surgical CompetencyOITE %tile YIT-20.2811560.000382 Surgical CompetencyOITE %tile YIT-30.2591540.001167 Surgical CompetencyOITE %tile YIT-40.1311370.125677 Surgical CompetencyOITE %tile YIT-50.1921170.038341 Surgical CompetencyUSMLE Step 10.0531310.547163 Surgical CompetencyUSMLE Step 20.018530.900569 Surgical CompetencyABOS %tile0.39650.001321 The only objective score to correlate with surgical competency was ABOS I percentile (medium correlation)

18 Survey/Score CategoryScore CategoryRSp - Value Overall CompetencyOITE %tile YIT-10.136890.202862 Overall CompetencyOITE %tile YIT-20.2311560.003719 Overall CompetencyOITE %tile YIT-30.2341540.003492 Overall CompetencyOITE %tile YIT-40.1041370.224354 Overall CompetencyOITE %tile YIT-50.1331170.154008 Overall Competency has no correlation with OITE YIT 1-5

19  Our “mildly blinded” study of 5 programs, objective measurements have very little correlation with a program director or chair’s opinion of a resident’s overall competency, core competencies outside of medical knowledge or surgical competency upon graduation.  The one exception is the ABOS percentile and surgical competency which is mildly correlative.

20 “When given a choice on who will operate on me or my children, with all other qualities being equal, I would choose the physician with the higher scores….” Steven Frick. June 2012


Download ppt "Steven Kane, MD David Robinson, MD Atlanta Medical Center."

Similar presentations

Ads by Google