Accreditation Council for Graduate Medical Education Milestones are Coming: A Conversation with the Family Medicine Milestones Committee May 2013

Overview Next Accreditation System Clinical Learning Environment Review Family Medicine Milestones

Why ‘Next Accreditation System’ (NAS)? “Self-regulation is a fundamental professional responsibility, and the system for educating physicians answers to the public for the graduates it produces.” “The Next GME Accreditation System – Rationale and Benefits” Nasca T.J., Philibert I., Brigham T., Flynn T.C. N Engl J Med 2012; 366:

Why ‘Next Accreditation System’ (NAS)? The ACGME's public stakeholders have heightened expectations of physicians. Patients, Payers, and the public demand -access to proper care and a good physician -information-technology literacy, -sensitivity to cost-effectiveness, -the ability to involve patients in their own care, and -the use of health information technology to improve care for individuals and populations.

Why ‘Next Accreditation System’ (NAS)? Evaluate programs based on reporting of outcomes through educational milestones which is the next step for the ACGME competencies. - Patient care - Medical Knowledge - Professionalism - Systems-based practice - Practice-based learning and improvement - Interpersonal and communication skills Foster innovation and improvement by programs.

Goals of NAS Strengthen resident development in Professionalism, Interpersonal & Communication Skills, Systems Based Practice, & Practice Based Learning Enhance public accountability More explicit definition of a good physician (Milestones) Patient safety is paramount (Clinical Learning Environment Review Program) Improve efficiency and reduce burden required for accreditation

NAS Overview Continuous oversight rather than episodic sampling Milestone reports (semi-annual), Resident & Faculty questionnaires (annually), Case Logs (annually), Program & Institutional Infrastructure (annually) monitored by ACGME Review Committees Program Site Visits at ~10 year intervals Emphasis on program effectiveness as evidenced by resident advancement through the milestones

‘Next Accreditation System’ in a Nutshell NAS allows accreditation process to advance from an episodic “biopsy” model to annual data collection. RRCs will measure compliance through the evaluation of annual program data elements.

Annual Program Data (Elements to Include) RC receives data ‘continuously’ RC tracks data on each program/residents Milestone Performance Resident Survey Results Faculty Survey Results Case Logs and other parameters of clinical experience Scholarly Activity Key Quality/Patient Safety Data Board Certification Examination Results Institutional/Fiscal/Faculty/Leadership, etc..

ACGME Strategy De-emphasize the current focus on the PIF and prescriptive program requirements Emphasize evaluation of program’s actual operations and implementation processes Enhance selected elements of visit - Review of citations - Resident complaints - Resident survey (non-compliance) - Duty hour and learning environment standards - Changes since last visit - Annual program evaluation

NAS Timeline June 2011 ACGME Board Approved NAS July 2012 Clinical Learning Environment Review (CLER) Visits Initiated December 2012 Draft Residency Milestones Completed July 2013 NAS begins in 7 Specialties NAS Policies & Procedures Finalized 2014 NAS Implemented in all Specialties 2015 Subspecialty Milestones Completed

Clinical Learning Environment Review Institutional CLER visits ~ every 18 months for each participating institution Entire first cycle of visits are beta testing and will not change accreditation Initial visits will be used for feedback, learning and to establish baseline information Site visits will be done by full-time staff and volunteer peers Six key areas that will be focused on during the site visits 12

Clinical Learning Environment Review Visits Integration of residents into institution’s Patient Safety programs and demonstration of impact Integration of residents into institution’s Quality Improvement programs and efforts to reduce Disparity in Health Care Delivery and demonstration of impact Establishment, implementation and oversight of Supervision policies Additional information available at:

Clinical Learning Environment Review Visits Oversight of Transitions of Care Oversight of Duty Hours Policy, Fatigue Management and Mitigation Education and Monitoring of Professionalism Additional information available at:

FM Milestones 14-member Committee Committee Meetings -March, July, October Conference calls between meetings Comment Period -Late 2012/Early Presentation at RPS/PDW Final Document – Summer 2013 Implementation – Summer 2014

FM Milestones Milestones are specific benchmarks of skills, knowledge, and behaviors that each resident is expected to achieve at identified stages of residency training. Milestones developed for each of the six ACGME competencies. Stages for the FM Milestones: Has not achieved level 1, level 1, level 2, level 3, level 4 and level 5.

FM Milestones Observable developmental steps describing a trajectory of progress from level 1 to level 5 Provide a “roadmap” for learning “Intuitively” known by experienced family medicine educators

FM Milestones Designed to allow tracking of discrete and measurable educational “outcomes” Build upon existing evaluation tools and observations of the faculty Additional evaluation tools and techniques may be developed, as needed Progress of each resident to be assessed by local program’s Clinical Competency Committee (CCC)

FM Milestones FM Milestones draft document will be on the ACGME website in February 2013 Feedback on the milestones encouraged Revised document presented at RPS/PDW for comment Final document – Summer 2013 Living document that will be updated

FM Milestones Each residency will have a clinical competency committee (CCC) CCC should include faculty only CCC should have at least 3 faculty members CCC should include faculty who are active in evaluation of the residents

FM Milestones CCC will review each resident’s progress in each competency and enter assessments on the milestones reporting form for each resident twice a year The program will provide feedback to each resident regarding their progress in each milestone

FM Milestones Residents do not need to achieve level 4 in every milestone to graduate but should substantially demonstrate the milestones targeted for this level as it is designed as the graduation target Residents do not need to achieve competency at level 2 in each milestone to advance to their second year of training but should achieve that level in the majority of milestones

FM Milestones Residents are not expected to achieve competency at level 5 during residency Residents may achieve a level of competency in specific milestones sooner than expected

FM Milestones Resident data will be de-identified on the semi-annual milestone form ACGME will compile data at program level and specialty level Milestone data will not affect accreditation cycle at this time

26

27

28

FM Milestones Feedback Common themes from general feedback: Too long - too short Too vague - too specific Hard stops or pass/fail that must be met Assessment tools Administrative burden

FM Milestones Alpha Testing 12 programs Recommended: 2 residents from each year – total 6 residents Variety of residents – top of class, bottom of class, middle of class Clinical Competency Committees Survey completed about the process

FM Milestones Alpha Testing CCC averaged 6 – 7 members Reviewed on average 6 residents Averaged time was 60 minutes/resident with range of 36 – 84 minutes 8/12 programs plan to modify their curriculum because of the experience 10/12 programs plan to modify or add new evaluation tools/methods

FM Milestones Alpha Testing Experience of programs

FM Milestones Revised milestones online: nas.org/assets/pdf/Milestones/FamilyMedi cineMilestones.pdf nas.org/assets/pdf/Milestones/FamilyMedi cineMilestones.pdf Programs to beta test the FM Milestones If interested, please send an to: with name of programs, name of program director and contact information

Accreditation Council for Graduate Medical Education Questions?