1. Milestone Project Update
Emergency Medical Services
January 17, 2014
Jane Brice MD, MPH
Professor
Laura Edgar EdD, CAE
Executive Director

2. Presentation Topics Milestone Background and Purpose Implementation
Clinical Competency Committees
Breakout Groups

3. THANK YOU Peter Pons, MD Jane Brice, MD, MPH Debra Perina, MD
Charles Beaudette, MD (Fellow)
Mike Beeson, MD
Nicholas Rathert, MD (Fellow)
Stacy Weisberg, MD
Felicia Davis
Noel Wagner, MD, NREMT-P
Anne Harvey, PhD

4. Milestones: What
Milestones describe performance levels fellows are expected to demonstrate for skills, knowledge, and behaviors in the six competency domains.
Milestones will lay out a framework of observable behaviors and other attributes associated with fellows’ development as physicians.
In the next accreditation system , aggregate fellow performance on the milestone level will be used as one indicator of a fellow’s educational effectiveness.
Nasca, TJ et. Al. The Next Accreditation System :

5. Key Points: Milestones
Articulate shared understanding of expectations
Describe trajectory from beginner in the specialty to exceptional resident or practitioner
Organized under six domains of clinical competency
Represent a subset of all sub-competencies
Set aspirational goals of excellence

6. Milestones: Why Fulfill the promise of the Outcome Project:
Increased use of educational outcome data in accreditation
ACGME accountability to public
Support the educational process

7. Uses and Implications Milestones ACGME
Accreditation – continuous monitoring of programs; lengthening of site visit cycles
Public Accountability – report at a national level on competency outcomes
Community of practice for evaluation and research, with focus on continuous improvement
Fellowship Programs
Guide curriculum development
More explicit expectations of fellows
Support better assessment
Enhanced opportunities for early identification of under-performers
Certification Boards
Potential use – ascertain whether individuals have demonstrated qualifications needed to sit for Board exams
Fellows
Increased transparency of performance requirements
Encourage fellow self-assessment and self-directed learning
Better feedback to fellows
There are four primary stakeholders in the Milestone Project: ACGME, Residency Programs, Residents, and Certification Boards. The CAGME will primarily use milestones for accreditation of GME programs. They will also use them for reporting to nation al interests and for research purposes. Residency programs may use the milestones to help guide curriculum and to aid residents. For residents that are not performing at expectation, the milestones become a clear means of explanation as to where the problems lie. Residents can use the milestones to do more self-assessment and to prepare for future learning. The results of a milestone evaluation will help the resident to understand their strengths and weaknesses. Each certification board will decide for themselves what they will do with this information. They may use it as a requirement for sitting for the board exams or as a launching point for maintenance of certification.

8. Milestone Template
This is the template the groups started with. The template demonstrates the expectations of the levels. Level 1 is for the beginning resident. The beginning resident is different for each specialty – some residents have come direct from medical school, some have had a transition year, some have even already completed another residency program. Level 4 is for the graduating resident. A resident who achieves a Level 4 is someone who is ready for certification and independent practice. Levels 2 and 3 are those steps in-between – these residents are continuing to learn. Level 5 is for those residents who exceed the expectations of the graduating resident.

9. EMS Milestones Group
Came together for a 1.5 day meeting We had representatives from the RRC, ABEM, NAEMSP and two recently graduated fellows Made decisions about framework for milestones and then got to work

10. EMS Milestones
Options for creation and organization of milestones – many to choose from:
EMS Core Content
EMS Curriculum
EMS Board Exam Outline
EMS Program Requirements
Fellowship Experience (both as a fellow and as a program director

11. EMS Milestones Organized under the 6 domains of Clinical Competence
Patient care
Procedures performed in the prehospital environment
Prehospital recognition and stabilization of time/life-critical conditions
Recognition and treatment of prehospital conditions
Mass casualty and disaster management
General special operations in EMS

12. EMS Milestones Medical knowledge Systems-based practice
6. Medical oversight
Systems-based practice
EMS personnel (supervision and training)
System management
Practice-based Learning and Improvement
9. Quality management
10. Research

13. EMS Milestones Professionalism Interpersonal Communication Skills
11. Ethics and professional behavior
12. Accountability
Interpersonal Communication Skills
13. Team communications and management
14. Patient-centered communications

15. Milestones are progressive over time
Milestones are progressive over time. There is no prescribed speed at which residents must complete a milestone set.
Selecting a response box in the middle of a level implies that milestones in that level and in lower levels have been substantially demonstrated.
Selecting a response box on the line in between levels indicates that milestones in lower levels have been substantially demonstrated as well as some milestones in the higher level(s).

16. When should we start?
NOW

17. What is the Clinical Competency Committee?

18. Clinical Competency Committee
Composed of a minimum of 3 faculty members
Non-physician members can be appointed
Reviews all evaluations by all evaluators semi-annually
Reviews residents against milestones semi-annually
Make recommendations for progress – promotion, remediation and dismissal

19. How does the CCC actually work?

20. Who should be on the CCC? Decision for PD Consider:
Representation from each major site
Subspecialty representation
Dedication to education

21. How do we DO the evaluation?
Understand the milestones & their use
Leave personal bias at the door
Review all evaluations for each resident
“Consider the source(s)”
For each resident, decide for each milestone the narrative that best fits that resident

22. The Clinical Competency Committee
Avoids common problematic issues:
“I don’t like to give negative evaluations”
“I spent little time working with this resident”
“Herd” mentality: positive or negative
Grade inflation
Vague statements:
“I just didn’t like this resident, but I can’t put my finger on it”
Hearsay: I’ve heard she is lazy

23. Clinical Competence Committee
Self
Evaluations
Case Logs
Mock Orals
End of Rotation Evaluations
Operative Performance Rating Scales
Unsolicited
Comments
ITE
Sim
Lab
Student Evaluations
Nursing and Ancillary Personnel Evaluations
Clinic Work Place Evaluations
Clinical Competence Committee
OSCE
Peer Evaluations
Patient / Family Evaluations
Assessment of Milestones
© 2012 Accreditation Council for Graduate Medical Education (ACGME)

24. Milestones: Reporting
All programs within a specialty use the specialty’s milestones
Programs will report semi-annually
Milestone data will be reported to ACGME through direct entry into ADS

25. Implementation Next Accreditation System for EMS starts July 1, 2014
First Milestone reporting is December 2014

26. MOCK CCC
Break into groups – members of the Milestone Group are here and can help answer questions
Evaluate Resident X against a 6 milestones
Share your experience with your faculty – excellent faculty training opportunity

27. Thank you!

28. Contact Information
Accreditation Questions: Felicia Davis Milestones Questions: Laura Edgar