J. L. Marsh, MD Chair - Orthopaedic RRC Director of the ABOS Carroll B. Larson Chair Residency Program Director University of Iowa Hospitals and Clinics Iowa City Iowa NAS - (and other new program requirements) Overview of what you will need to do!

 Next Accreditation System (NAS)  Milestones  Procedural minimums  Continuous data assessment  Surgical skills training  Greater institutional oversight  No PIF, less and different site visits We are about to experience tremendous changes to the oversight of GME The financial implications of these changes are uncertain at this time

Traditional Program Review  Program review scheduled  PIF prepared and sent to ACGME and SV  SV – 1-2 days  RRC review  PIF and SVR  Board pass rates, Resident Survey, Case log data  RRC actions  Initial or continued accreditation with citations  1-5 year cycle  Progress report  Propose probation SV and PIF are key portions of program review

What is NAS??? Program accreditation system without: –Mandated site visits –PIF’s –100% mandatory PR’s –Direct resident interviews Program accreditation system with: –Annual or semi annual data review –Performance metrics within the data elements –Emphasis on outcomes –Focused site visits driven by data metrics –More opportunity for program innovation –Data charts, graphs and data flows!!! Notice the word “data” is used 5 times!! Notice the “PIF and site visit” are going away

NAS is not all about Milestones  Milestones will be a work in progress but will not be a part of program accreditation for several years  But Milestones are one of the new requirements where increased effort and cost for programs maybe necessary and they need to start now!!

NAS represents a substantial change in program oversight  Change in focus / function of RRC  More educational  Less regulatory  PD’s empowered to innovate & create an excellent program  Core vs. detail requirements  Improved tools for program review without a PIF  Focused reviews triggered by parameters set by the RRC

The data for Program Review by RRC  Trended & weighted performance metrics –Program data –Resident and faculty scholarship (new template) –Clinical experience (enhanced case logs) –Resident Survey (new questions) –Core Faculty Survey (new) –Semi-annual Resident Evaluation Milestones (new) Clinical Competency Committee (new) –Rolling Board pass rates (Parts I & II) –Program Self-Study (new) Site Visit (every 10 years) Many of these are outcomes, many of them are new!

Performance thresholds based on data elements Weighting of data elements will provide screening criteria RRC annual review and action only if necessary Potential actions include:  Initial or Continued Accreditation  Request more information from program  Request Site visit (focused or full) on short timeline  Continued Accreditation with Warning  Probation “If there is a problem get in there and fix it” “If the data is good…….leave them alone…innovation” Absent – SVR and PIF document

Program Requirement Changes Common & Specialty Specific (no change) Core: requirement must be met as specified; if not, program can be cited Detail : programs will not be assessed for compliance with these requirements if they demonstrate good educational outcomes. These are mandatory for new programs & those that failed to meet outcomes expectations (on Probation or Continued Accreditation with Warning) Outcome – Some data elements are based on these

NAS Program Review  Each program reviewed at least annually  NAS is a continuous accreditation process –Review of annually submitted data –Supplemented by: Reports of self-study visits every ten years Progress reports (when requested) Reports of site visits (as necessary)

 Annual Review of Data (Oct. – Nov.)  Options Available Prior to January RRC Meeting Focused Site Visit Full Site Visit Request Clarification or Progress Report Send Material out to RRC for Review Highlighting the Problem(s) for Peer Decision Move to Consent Agenda Proposed workflow prior to RRC meeting

Minimal notification given Minimal document preparation expected Team of site visitors Specific program area(s) investigated as instructed by the RRC NAS: Focused Site Visit

Application for new program At the end of the initial accreditation period RRC identifies broad issues / concerns Other serious conditions or situations identified by the RRC NAS: Full Site Visit

 Programs (CA) meet all established performance indicator thresholds (40%) – letter from ED Continued accreditation with no RRC review  Programs (CA) fail to meet 1 established performance indicator (30%) - letter from ED Continued Accreditation but notes need for improvement - indicates the deficiency  Programs that fail to meet 2 established performance indicator thresholds but not “High Stakes” indicators (20%) – ED reviews program for trends – if first time event letter from ED Continued Accreditation notes problems no further RRC review Theoretical Work Flow – Consent agenda (90%)

 Programs fail to meet 3 -5 established performance indicator thresholds (7%) – Two RRC reviewers assigned  Programs fail 6-9 performance indicator thresholds (3%) – Assigned a focused or full site visit Theoretical Work Flow – RRC review

For NAS you need to do two things! Do well on the performance metrics Appoint a new committee to oversee Milestones and develop a plan to evaluate them

The data for Program Review by RRC  Trended & weighted performance metrics –Program data –Resident and faculty scholarship (new template) –Clinical experience (enhanced case logs) –Resident Survey (new questions) –Core Faculty Survey (new) –Semi-annual Resident Evaluation Milestones (new) Clinical Competency Committee (new) –Rolling Board pass rates (Parts I & II) –Program Self-Study (new) Site Visit (every 10 years) Many of these are outcomes, many of them are new! Maintain a consistent solid performance on all of these!!

Clinical Competency Committee Semi Annual Review of Data to assign Milestones CCC - Faculty time and input necessary for these individual resident evaluations which are the Milestones There may also be a PEC committee. – program evaluation

Milestones 5 level assessments of resident knowledge, skills, attitudes, and other attributes of performance in the six competencies in a developmental framework from less to more advanced. They are designed to demonstrate program outcomes by assessing resident progress through the competencies measured in the milestone framework!

Milestones : Medical Knowledge & Patient Care ACL Ankle Arthritis Ankle Fracture Carpal Tunnel Degenerative Spine Diabetic Foot Diaphyseal Femur & Tibia Fracture Distal Radius Fracture Adult Elbow Fracture Hip & Knee Osteoarthritis Hip Fracture Metastatic Bone Lesion Meniscal Tear Pediatric Septic Hip Rotator Cuff Injury Pediatric Supracondylar Humerus Fracture Small slices of clinical care – a biopsy of resident performance!

Milestones: Medical knowledge (example)

Operationalizing Milestone reporting? The faculty, PD and PC time and effort to accomplish this remain uncertain Therefore the tradeoff for absence of SV’s and PIF’s remains uncertain In my opinion they are good assessments which will make a more uniform national standard to assess resident competence

There are other non NAS requirements that will have financial implications for your department 6 months of PGY 1 ortho Mandated surgical skills training through simulation

PG-1 Year Changes ABOS certification rules developed from results of a CORD survey ACGME/RRC accreditation rules developed from ABOS  6 months of orthopaedic surgery  Basic surgical skills training Good news – they are the same!

PG-1 Year Changes More time on orthopaedics!  So orthopaedic PGY 1’s will be on ortho for 6 months instead of 3 months  In our program we have 6 PGY 1’s so effectively this is a junior level 1.5 FTE  How much of a cost advantage for a department is this?  6 months of orthopaedics

Basic surgical skills requirements (core)  A curriculum with goals and objectives  Assessment metrics  A dedicated space for the skills training  Training in basic skills required of residents for emergency care and to prepare residents for future participation in surgical procedures This is what is required! What will that cost?

Results of a 2011 National Orthopaedic Program Director and Resident Survey – Karam and Marsh JBJS 2012  Only 50% of residency programs have a skills lab and program.  There is high interest among PD’s in a skills curriculum.  Most PD’s have little knowledge of the budget for skills training or the cost of a skills lab  Cost is a challenge to expansion of skills programs

Interest in a curriculum?

*Percentages may not total 100% because respondents were allowed to choose more than one answer. Lack of funding Barrier to skills program

Members of the ABOS (AOA/CORD and AAOS ) Surgical Skills Task Force J. Lawrence Marsh, MD – Chair (ABOS) James E. Carpenter, MD (ABOS) Shepard R. Hurwitz, MD (ABOS) Michelle A. James, MD (ABOS) Joel T. Jeffries, MD (AOA/CORD) David F. Martin, MD (ABOS) Peter M. Murray, MD (ABOS) Bradford O. Parsons, MD (AAOS) Robert A. Pedowitz, MD, Ph.D. Co- Chair (AAOS) Brian C. Toolan, MD (AAOS) Ann E. Van Heest, MD (AOA/CORD) M. Daniel Wongworawat, MD (AAOS)

1.Sterile technique and operating room set up 2.Knot tying & suturing 3.Microsurgical suturing 4.Soft tissue handling techniques 5.Casting and splinting 6.Traction 7.Compartment syndrome 8.Bone handling techniques 9.Fluoroscopy 10.K-wire techniques 11.Basic techniques in ORIF 12.Principles and techniques of fracture reduction 13.External fixation 14.Basic Arthroscopy skills 15.Basics of Arthoplasty 16.Joint injection 17.Patient Safety Modules (ABOS skills taskforce modules)

Modules should include: Low cost low tech options

Modules should include: Evaluation and assessment strategies Guided practice until performance within time standards Video of performance with blinded review by expert faculty with “pass” or “needs more practice” OR performance ONLY after verification

January 2013 All 6 PGY 1’s Some call on weekend no other clinical work

Summary and Conclusions  Resident satisfaction was high.  A dedicated month of surgical simulation has potential to change the paradigm of skills training for junior residents.  Considerable time invested in the planning and execution but faculty members were eager to contribute.  The greatest expense was for cadaveric specimens. With better planning more cost effective simulations, this expense could be reduced.

Overal Summary and Conclusions  NAS will take more time/effort and more yearly cost  Less demanding in the year of a site visit  The balance is hard to know  Other changes to PR at PGY 1 will affect finances  Less availability for all night call  More ortho time for PGY 1’s (3 vs 6 months)  Surgical simulation will cost money  More or less depending….  This investment may be worthwhile