Presentation on theme: "Measuring Outcomes for Residency Graduates Steven L. Frick, MD Chairman, Dept. of Orthopaedic Surgery Director of Medical Education, Nemours Children’s."— Presentation transcript:
Measuring Outcomes for Residency Graduates Steven L. Frick, MD Chairman, Dept. of Orthopaedic Surgery Director of Medical Education, Nemours Children’s Hospital Professor and Assistant Dean University of Central Florida College of Medicine Orlando, Florida
No financial disclosures. AAOS Program Committee POSNA Curriculum Committee POSNA Residents Review POSNA Treasurer, Board of Directors JRGOS Board of Directors ABOS QWTF ACGME Milestones Project Workgroup
No competency specifically addressing surgical skills
Macy Foundation Report 2011 The Content and Format of Graduate Medical Education Recommendation III-A: The length of GME should be determined by an individual’s readiness for independent practice- demonstrated by fulfillment of nationally endorsed, specialty-specific standards- rather than tied to a GME program of fixed duration.
“nationally endorsed, specialty- specific standards” Do we have any of those?
Role for CORD Optimistic versus Pessimistic “an opportunity in every difficulty” versus “ a difficulty in every opportunity”
Evidence Based Medicine Integrating individual clinical expertise with the best external clinical evidence Outcome A final product or end result
A. Flexner Medicine can be learned but not taught Active participation required Need dedicated educators and students
“get comfortable with uncertainty”
Role of professional education Provide practitioners the intellectual tools to assess information critically, stay abreast of changing knowledge, adapt to continuous change, and reflect on the larger role and responsibilities of the profession in society. From Time to Heal by Kenneth Ludmerer
“Is there a core body of knowledge and skills that the finishing resident should possess prior to starting practice or fellowship?” - Richard Gross, MD Need curriculum and competency assessment
Residency fundamentally = Master-Apprentice
William Halsted: Residency Training system Introduced in 1889 at Johns Hopkins based on: a fixed period of time for training, structured educational content, actual experience with patients, escalating responsibility for patient care during training, and a period of supervised practice after formal training. Remains the cornerstone of surgical training in North America more than a century later
Competency Based Education Defined by the outcome of the educational process, not the content Develop weighted curriculum to teach and assess (Farmer, Gross, Wadey) Assessing competence focuses on what the learner is able to do
How do you assess competency? "the state or quality of being capable or competent; skill; ability."
Miller’s model of competence Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7. Professional authenticity Read, Listen Knows Shows how Knows how Does Performance or “hands on” Live Demo; Multimedia
Dreyfus Model of Skill Acquisition
Mastery Learning Model -Bloom 1968
Becoming Expert “The 10,000 Hour Rule” About 10 years- dedicated practice Bill Joy- UNIX, Sun Microsystems; Mozart; The Beatles; Bobby Fischer;Bill Gates
Model of complete clinical care Engage Empathize Educate Enlist Fix It Find It Opening Closing
Culturally Competent Care The ability to understand and work with patients whose beliefs, values, and histories are significantly different from our own.
Quality of OutcomesQuality of Outcomes Patient-Physician RelationshipPatient-Physician Relationship Malpractice ClaimsMalpractice Claims CCC Education Team Harmony & Quality of Work Life Error Prevention Medical/ Surgical Team Concerns Cultural Competence in Health Care Courtesy of A. White, III, MD
Defining / Teaching/ Modeling Professionalism most important Drs. Cruess body of work Hidden curriculum Social Contract Individual Awareness
Teach and Model Professionalism
Surgery- tripartite body of knowledge Frank Wilson, MD Preoperative - evaluation, indications, planning Intraoperative - technical execution Postoperative - immobilization, weight-bearing, PT All 3 necessary for success
Ortho Surgical Education Interns - pre and post operative care, framework of ortho fundamentals, closed management of fxs PGY 2/3 - basic decision-making and psychomotor skills PGY 4/5 - independent decision-making, subspecialty skills, integrate knowledge
Our Educational philosophy at CMC Not training Stimulus - Reaction vs Stimulus - Thought - Reaction Create one-on-one master-apprentice situations Graduated responsibility ALWAYS supervised in highest risk activities (OR) Have to spend enough time with them to know
I DON’T KNOW How do you assess competency?
Charlotte Competency Stages Stage I - do not know anything cannot do anything, and know it Stage II – know and can do a lot, but do not recognize what you do not know and cannot do DANGER Stage III – know and can do a great deal, but realize there is much you do not know
“The beginning of a mountaineer’s career, when energy and enthusiasm outpace experience and judgment, is said to be the most dangerous part.” Photo by Guillaume Dargaud
Setting Standards Job of Chair and RPD to set standards of excellence Graduates of program should meet these standards in all core competencies
Assessing competency Complete 5 years of orthopaedic surgery program under watchful eye of PD 12 months PGY1 / internship, 48 months orthopaedic surgery Evaluations and comment by faculty, peer evaluations, portfolio (presentations, courses, outcomes instruments), OITE, operative experience log Consensus of PD and faculty
ABOS I believe this individual is capable of the competent independent practice of orthopaedic surgery. Steven L. Frick, MD Residency Program Director
An Expert- Knows Knows WHAT to do Knows HOW to do it Knows WHEN to do it (and when not to) Knows WHY to do it Knows WHEN to ask for help Knows WHAT we don’t know
Is it possible/desirable to define and measure competency and then graduate a resident before 5 years?
A Competency-Based Curriculum in Orthopaedic Surgery: From Idea to Implementation Markku Nousiainen, MS, MD, MEd, FRCS(C) Sunnybrook Health Sciences Centre University of Toronto
Current challenges in residency training reduction in work hours reduced time spent in OR teaching surgical skills reduction in wait times improvement in patient safety reduced training opportunities for residents }
Competency-based education “Training process that results in proven competency in the acquisition & application of skills & knowledge to medical practice that is not simply dependent on the student’s length of training & clinical experiences”
“Much of what is counted does not count, and much of what counts cannot be counted.”
Problems Toronto experience- 5 years, now all in for first time Still no defined “curriculum” More resource intensive than traditional pathway = costs more Current environment of GME= very dependent on Medicare funding Some predict reduction in Medicare GME funding under PPACA 2010
How much of residency education is experiential? Can we list / define everything you need to learn? Can we transfer knowledge gained from experience without making residents have the experience?
Duty hours 2003 First ortho class with 80 hour work week- double failure rate on part I ABOS certification exam Similar result 2011 exam takers Why? Does this exam measure competency? Who do you want – 90%ile or 30%ile?
GME-Decade of Accountability To patients by residents, faculty Patient safety, Resident safety- RPD To residents by faculty, institution Societal demands for assurance of competency Safe, Effective, Patient centered, Timely, Efficient, Equitable (IOM) Increased requirements by oversight organizations – RPD time Professional, ethical behavior demanded
NAS- Next Accreditation System Coming to Ortho July 2013 No more site visits, PIFs every 5 years Annual “Biopsy” of 4 things –Institutional report –Annual survey of residents and faculty –Case logs –Milestones (q 6 mos reports from Competency Committee) Self report every 10 years
Ortho Milestones- 18 cover PC and MK All have 5 levels By graduation resident should be level 4 (competent) in all For peds- septic arthritis and SC humerus fracture Surrogates for knowledge in other areas
Milestone- Peds SCH Fracture
NAS Institutional reviews (q 18 mos) Milestones reports (q 6 mos)- form a competency committee Operative experience database Resident annual survey Faculty annual survey (new) ORTHO JULY 1, 2013
Future of Orthopaedic Residency Education Change is coming Need to protect experience, in addition to more rigorous evaluation / oversight More evaluations / structured experiences Remember importance of graduated independence Milestones will be modified as we go NAS is on the way- BE AN OPTIMIST!