Presentation is loading. Please wait.

Presentation is loading. Please wait.

Steven Kane, MD David Robinson, MD Atlanta Medical Center.

Similar presentations

Presentation on theme: "Steven Kane, MD David Robinson, MD Atlanta Medical Center."— Presentation transcript:

1 Steven Kane, MD David Robinson, MD Atlanta Medical Center

2  Core Competencies  Prior USMLEs  Interviews  Faculty Evaluations  Intra-operative observation  Clinical observation  Surgical Skills lab  Peer to Peer  Pre medical experience  Ancillary staff evaluations  OITE  Milestones

3 9.63


5  “I shall not today attempt further to define the kinds of material I understand to be embraced within that shorthand description and perhaps I could never succeed in intelligibly doing so. But I know it when I see it……” 1964 Potter Stewart

6  Carmichael, Kelly D. MD; Westmoreland, James B. MD; Thomas, John A. MD; Patterson, Rita M. PhD  May 2005, Southern Medical Journal  The only statistically significant correlations to OITE scores were USMLE step I performance (220) and marital status (married)

7  Paul J. Dougherty MD; Norman Walter MD; Peter Schilling MD; Soheil Najibi MD, PhD; and Harry Herkowitz MD  March 2010, Clinical Orthopaedics and Related Research  Those who averaged in the 27th percentile or lower on the OITE had a 57% chance of failing the ABOS Part I examination  Correlation coefficient of 0.53 between the USMLE Step 1 and ABOS Part I  Average correlation coefficient of 0.50 for postgraduate years (PGY) 2 through 5 OITE scores and ABOS Part I

8  KLEIN, GREGG R. MD; AUSTIN, MATTHEW S. MD; RANDOLPH, SUSAN; SHARKEY, PETER F. MD; HILIBRAND, ALAN S. MD  May 2004, The Journal of Bone and Joint Surgery-American Volume  Correlation coefficient for the USMLE Step-I score and the ABOS Part-I percentile score was 0.38 (p = 0.002)  Correlation coefficients were also calculated for the OITE percentile scores for years 3 (r = 0.52, p < ), 4 (r = 0.49, p < ), and 5 (r = 0.69, p < ).  There was a high risk of failure (63%; five of eight failed) on the ABOS Part-I examination when a resident scored below the twenty-ninth percentile for PGY-3 and below the twentieth percentile for PGY-5

9  5 programs evaluated 176 current and former residents evaluated  Chairs or PD’s asked to rate overall competence and surgical competence along with the 6 core competencies while “modestly blinded” to USMLE, OITE, and ABOS percentiles


11 A statistical cross comparison of USMLE 1, USMLE 2, OITE 1-5 percentiles, 6 Core Competencies, ABOS percentiles, Surgical Competency and Overall Competency upon graduation was compiled.


13 Variableby VariableN Spearman ρ Prob> │ρ│ ABOS Part 1 PercentileOITE ABOS Part 1 PercentileOITE <.0001 ABOS Part 1 PercentileOITE ABOS Part 1 PercentileOITE ABOS Part 1 PercentileOITE <.0001

14 Correlation with ABOS Survey CategoryNSpearman ρ Prob> │ρ│ Patient Care Medical Knowledge650.54<.0001 Practice-Based Learning and Improvement Interpersonal and Communication Skills Professionalism Systems-Based Practice Surgical Competency Overall Competency

15 Survey/Score CategoryScore CategoryRSp - Value Medical KnowledgeABOS %tile OITE %tile YIT-5ABOS %tile OITE %tile YIT-2ABOS %tile Surgical CompetencyABOS %tile Medical KnowledgeOITE %tile YIT Overall CompetencyABOS %tile Medical KnowledgeOITE %tile YIT Medical KnowledgeOITE %tile YIT OTIE %tile YIT-3ABOS %tile USMLE Step 2ABOS %tile ABOS was more correlative with surgical and overall competency than OITE (medium correlation)

16 Survey/Score CategoryScore CategoryRSp - Value PBL & ImprovementABOS %tile Systems-Based PracticeABOS %tile ProfessionalismUSMLE Step Interper. & Comm. SkillsUSMLE Step Surgical CompetencyOITE %tile YIT Interper. & Comm. SkillsABOS %tile Medical KnowledgeOITE %tile YIT Surgical CompetencyOITE %tile YIT OITE %tile YIT-4ABOS %tile Overall CompetencyOITE %tile YIT OITE YIT- 4 Appears to be a throw away year

17 Survey/Score CategoryScore CategoryRSp - Value Surgical CompetencyOITE %tile YIT Surgical CompetencyOITE %tile YIT Surgical CompetencyOITE %tile YIT Surgical CompetencyOITE %tile YIT Surgical CompetencyOITE %tile YIT Surgical CompetencyUSMLE Step Surgical CompetencyUSMLE Step Surgical CompetencyABOS %tile The only objective score to correlate with surgical competency was ABOS I percentile (medium correlation)

18 Survey/Score CategoryScore CategoryRSp - Value Overall CompetencyOITE %tile YIT Overall CompetencyOITE %tile YIT Overall CompetencyOITE %tile YIT Overall CompetencyOITE %tile YIT Overall CompetencyOITE %tile YIT Overall Competency has no correlation with OITE YIT 1-5

19  Our “mildly blinded” study of 5 programs, objective measurements have very little correlation with a program director or chair’s opinion of a resident’s overall competency, core competencies outside of medical knowledge or surgical competency upon graduation.  The one exception is the ABOS percentile and surgical competency which is mildly correlative.

20 “When given a choice on who will operate on me or my children, with all other qualities being equal, I would choose the physician with the higher scores….” Steven Frick. June 2012


Download ppt "Steven Kane, MD David Robinson, MD Atlanta Medical Center."

Similar presentations

Ads by Google