1. Milestones, EPAs, NAS…and Other ACGME Jargon
Committee on Graduate Medical Education
September 24, 2012
Sara LP Ross, MD

2. Objectives
To discuss the Next Accreditation System and what is known about how that will look in 2019
To discuss the system of trainee evaluation – Milestones and EPAs

3. NAS: Next Accreditation System
Goals
Complete realization of the Outcomes promise
Free up “good programs” to innovate
Assist poor programs in improving
Reduce burden of accreditation
Establish and implement milestones to better track program and institutional performance
Provide accountability to the public
July 2012: Seven initial core specialties/RRCs begin NAS training
July 2013: NAS officially begins; seven specialties “go live”; remaining specialties begin training
July 2014: All specialties/RRCs using NAS

4. NAS: Next Accreditation System
10-year self-study visit model: next visit 2019 for all Peds department programs
Structure, Resources, Core processes, Detailed processes, Outcomes
“What is your plan for the next 10 years to improve”
Residents will submit a confidential consensus list of five strengths and “opportunities for improvement” (OFIs) the residents wish to discuss
Site visitor will share strengths, but will only share OFIs if residents give permission (makes residents feel more connected to the site visit)
Annual program surveillance
Performance indicators for each specialty developed by a “community of educators” within the specialty
Annual Resident Survey
Core elements of the competencies
Levels of performance
Core methods of assessment
Annual Faculty Survey
Case Log Data
Acgme will define the components of the self-study in preparation for the site visits over the next few years

5. NAS: Next Accreditation System
Institutional 18 month review
Ongoing creation of Milestones
Programs will get a letter annually stating that they meet all “performance indicator thresholds”
May be placed on “accreditation with warning” at any point during the 10-year cycle
May warrant an “immediate visit”
If problems not fixed during a given window of time, program may be placed on probation

6. Site Visits Structure of the visit: 2 site visitors
Brief meeting with PD
Resident and faculty interviews
Meeting with DIO
Meeting with PD
PIF Elimination – YES (most likely)
More focus on strengths of programs

7. Competencies
Competence: the ability to do something successfully (Oxford Dictionary of English)
Competencies: broad, general attributes of a good doctor
With attempt at evaluation they get widdled down to detailed skills/activities
In the end don’t really reflect the original meaning of the general competency
Competence = Attribute
Activity= Element of professional work
Both competence and competency should be thought of as a “quality or state of being”
- For example if the trainee should have the competencies to handle the care of a well newborn, then the next question should be “and what are these competencies?”
i.e. checking the red reflex on exam, verifying administration of Vitamin K, assessing maternal risk factors
Competent professional is clearly more than the sum of many detailed operational competencies
Most activities reflect several competencies and most competencies are applied in several activities

8. Entrustable Professional Activities (EPAs)
Units of work that may be awarded a more or less formal qualification at the moment when supervisors confirm the trainee is ready to assume responsibility for such activities
Not ceremonial; can happen at any moment
STAR: implies that a trainee has demonstrated enough competence to carry out the activity in question independently from now on
- Requires supervisors to make deliberate decisions about trainees competence to perform all critical activities at some point in time

9. Entrustable Professional Activities (EPAs)
Which critical professional activities cover the relevant competencies of the profession?
How can supervisors learn when to entrust such activities to the trainee?
“Trust reflects a dimension of competence that reaches further than observed ability. It includes the real outcome of training – the quality of care”
Part of essential professional work
Require specific knowledge, skill and attitude
Lead to recognized output of professional labor
Confined to qualified personnel
Be independently executable within a timeframe
Be observable and measurable in its process and outcome (well done or not well done)
Reflect one or more competencies to be acquired
Authors estimate EPAs should cover the objectives of a full postgraduate training program of 5-6 years

10. Domains of
Competency
Competencies
Milestones
EPAs

11. EPA/Competency Matrix
Viewpoint: Competency-Based Postgraduate Training: Can We Bridge the Gap between Theory and Clinical Practice?
ten Cate, Olle; Scheele, Fedde
Academic Medicine. 82(6): , June 2007.
DOI: /ACM.0b013e c7

12. EPAs
May be acknowledged formally as a “statement of awarded responsibility” (STAR)
Five levels of proficiency
Has knowledge
May act under full supervision
May act under moderate supervision
May act independently
May act as a supervisor and instructor

13. Expected Levels of Confidence
Time to achieve STAR in a specific EPA dependent on:
The EPA
The working environment
The trainee
The clinical teacher
Columns are pgy2, 4 and 6

14. EPA Mapped to Competencies/ Subcompetencies

16. Milestones
Developmental roadmap for the competencies and subcompetencies
Observable developmental steps moving residents from novices to experts/masters
Means of restructuring competencies into a measurable rubric of six domains of clinical competency

17. Domains of
Competency
Competencies
Milestones
EPAs

18. What milestone levels equate to different levels of proficiency?
Who determines you can practice the EPA independently going forward?
Minimum standards for advancement/graduation?

19. Where Are We in Pediatrics?
January 2012 The Pediatrics Milestone Project (51)
September 2012 Selection of 21 Pediatric Milestones to be
reported on semi-annually
- Requests for additional sites to study/ develop each of these milestones
Fall/Winter 2012 Program development of evaluation tool
July Implementation of Milestone reporting
51 milestones across 7 acgme competency domains; the additional domain being personal and professional development; not utilized in the ultimate 21 selected for evaluation

21. Where Are We in Pediatrics?
January 2012 The Pediatrics Milestone Project (51)
September 2012 Selection of 21 Pediatric Milestones to be
reported on semi-annually
- Requests for additional sites to study/ develop each of these milestones
Fall/Winter 2012 Program development of evaluation tool
July Implementation of Milestone reporting
Evaluation algorithm: envisioned by ACGME that in the initial phase of implementation/evaluation, programs will use typical evaluation tools and a small committee of faculty will utilize those evaluations to completed the milestone grids; estimated to take about an hour/resident the first go-round and faster thereafter

22. References
Carraccio C, Burke A. Beyond competencies and milestones: Adding meaning through context. J of Grad Med Ed. 2010;2(3):
ten Cate O. Trust, competence, and the supervisor’s role in postgraduate training. British Medical Journal. 2006;333:
ten Cate O, Scheele F. Competency-based postgraduate training: Can we bridge the gap between theory and clinical practice? Acad Med ;82: