1. Oversight of Underperforming Programs Through Special Reviews
Some slides adapted from Tia O. Drake, Exec Dir. GME
Washington University SOM

2. Who we are …. • The Graduate Medical Education Committee
Saint Louis University School of Medicine
Office of Graduate Medical Education
• Sponsor of 63 ACGME Accredited Training Programs
• Employ ~ 630 residents and clinical fellows

3. What We Do
Saint Louis University’s Graduate Medical Education Committee functions as the oversight committee for the School of Medicine’s residency, subspecialty and fellowship programs.
Establishment of institutional policies for graduate medical education as detailed in the institutional requirements. Establishment and maintenance of appropriate liaison with residency directors and with the administrators of other institutions participating in programs sponsored by the institution.
Regular review all residency training programs in relation to their compliance with institutional policies and the policy and curricular requirements of the relevant ACGME review committee.
The GMEC must assume an educational environment in which residents may raise and resolve issues without fear, intimidation or retaliation. .
Establishment and implementation of policies that affect all residency programs regarding the quality of education and the work environment for residents in each program, including monitoring duty hours and other work environment features as concerns are brought to the committee.

4. Old Accreditation System
• Provided 3 -5 year accreditation cycles
• Accreditation Status remained consistent for the
duration of the accreditation cycle
• Midpoint internal reviews allowed time for programs to
make adjustments prior to the next site visit
• Requirements were less demanding therefore there was
(theoretically) more time for institutional oversight

5. Old Accreditation System
The documentation of the Program Evaluation Committee (PEC)
if any, was retained within the training program
Program documentation was varied
Action plans were not required
Annual RRC updates had no immediate impact on the program or
institution

6. N Engl J Med Mar 15;366(11):1051-6

7. Continuous accreditation model; No cycle lengths

8. ACGME New Accreditation System (NAS)
• Continuous Accreditation System
– 10 year self study process
– Annual ACGME updates and review - citations will be levied by RRCs
– ACGME Surveys used to identify:
New citations
• Concerning Trends
• Areas for Improvement (AFI)
• Program Citations
– Reviewed annually-– Concerning trends may become NEW citations
- Citations can be removed as early as the next RRC meeting

9. New Accreditation System
• Program statuses are updated and could change annually based on
RRC review
Existing Programs can receive New Programs can receive
Continued accreditation Initial accreditation
Continued accreditation with warning Initial accreditation with
Probationary accreditation warning
Withdrawal of accreditation Accreditation withheld
Accreditation with Warning not appealable
– Accreditation is viewable on the open ACGME website
• Other more ominous RRC actions include: Focused site visits
– Formal adverse actions (probation, withdrawal, reduction in
resident complement) AND, all of these require GMEC action.

10. Institutional Requirements on Special Reviews
• I.B.6. The GMEC must demonstrate effective oversight of
underperforming program(s) through a Special Review process. (Core)
• I.B.6.a) The Special Review process must include a protocol that:
(Core)
– I.B.6.a).(1) establishes criteria for identifying underperformance; and, (Core)
– I.B.6.a).(2) results in a report that describes the quality improvement
goals, the corrective actions, and the process for GMEC
monitoring of outcomes. (Core)

11. Previously our process relied heavily on the Institutional Annual Program Evaluations (APEs)

12. Going Beyond the APE to the Special Program Review Process
As a result of our ACGME Institutional visit on January 16, 2018, we are now
required to expand our program review data and to significantly expand our
special review processes and oversight.

13. Our Special Review Criteria are found in GME policy 2.4
Evidence of underperformance or non-compliance with ACGME standards that are considered by the GMEC to risk a program’s accreditation status may prompt a Special Review. This includes, but is not limited to any of the following:
Deviations from expected results in standard performance indicators:
  Resident evaluation of the program Clinical Experience
Faculty evaluation of the program Loss of major participating site
Board pass rate Resident or Faculty Survey
Milestones Competencies
Case log volumes measured by:
Any senior resident graduating without meeting 100% of
requirements

14. Our Special Review Criteria are found in GME policy 2.4 (cont.)
• GME APE Dashboard results, including declining performance over most recent 2 or more years
• RRC notification to the program of citations or warnings considered by the GMEC to risk
accreditation status
• Major change in the program or department which may affect accreditation status
• Request by a program director for review of his/her own program
A program’s inability to demonstrate success in any of the following focus areas:
Integration of residents/fellows into institution’s Patient Safety Programs;
Integration of residents/fellows into institution’s Quality Improvement Programs and
efforts to reduce Disparities in Health Care Delivery;
Establishment and implementation of Supervision policies;
Transitions in Care;
Duty hours policy and/or fatigue management and mitigation; and
Education and monitoring of Professionalism
Communications about or complaints against a program indicating potential egregious or
substantive noncompliance with the ACGME Common, specialty/subspecialty-specific Program,
and/or Institutional Requirements; or noncompliance with institutional policy.

15. Types of Special Reviews
Special reviews may be triggered by notification from the ACGME
including Self-Study, New Program and For Cause site visits, and by recommendation of the GMEC and/or DIO based on serious concerns or program data that may impact the program or its accreditation. Full or modified special reviews also may be done at the discretion of the GMEC and/or the DIO for an underperforming program.
Key Elements of Special Reviews
The Special Review team members are provided:
• A copy of the Institutional APR letter detailing areas of concern
• Most recent ACGME and Institutional Anonymous Surveys
• Any other relevant documentation identified during the recent APR

16. Team Members Selection and Duties
Review teams, selected and requested by GME staff, are composed of a senior GMEC member (preferably a Program Director) who has previously conducted a review, a GMEC member who has not previously conducted a review, a Resident member and GMEC staff.
Teams will conduct separate meetings with
(1) PD and PC*
(2) Faculty
(3) Residents
Teams (or GME staff) will conduct a document review
*Guidance for these meetings are found in GME policies 2.4b-e.

17. The Team Report Content and Approval Process
Following completion of the data gathering meetings and reviews, Teams will prepare a report using Special Review Template 2.4f to include remedial actions, timelines, and monitoring plan. These reports will then be submitted to the GMEC for review, approval and monitoring assignment.

18. The Benefits of Special Reviews
Program Under Review:
• Helps to open lines of communication between the Program Director, Program Coordinator , faculty and house staff
• Assures program has taken steps to correct concerns
• Helps to educate Teaching Faculty and House Staff on current
RRC expectations
• Determine whether the program can document compliance
with requirements

19. Remediation Plans
Significant issues may warrant a follow up meeting with:
– Core PD
– Coordinator
– Chair
– Division Director
–– Hospital and/or University
Administrative leaders