1. Robert Englander, MD,MPH UCSF February 26th, 2015
EPAs as a Framework for Assessment Across the Continuum
Robert Englander, MD,MPH
UCSF
February 26th, 2015

2. Disclosures
I have no financial Conflicts of Interest to disclose

3. Objectives
Develop a working knowledge of EPAs and their relationship to competencies and milestones
Apply that working knowledge to a specific example of a handover communication EPA
Make the connections between EPAs across the Education and Training Continuum

4. Setting the Stage We have milestones for 49 competencies
How do they contribute to meaningful assessment?
Inform us about learners at a granular level as we directly observe them
Provide substrate for formative feedback and a learning roadmap
BUT…

5. Setting the Stage
Need assessment tools to inform judgments about level of milestone performance
Need to integrate the competencies & their milestones to meaningfully assess whether learners can put it all together to deliver care
AND
Need to cluster the milestones to make assessment both meaningful and practical

6. EPAs: Using The Milestones as “Building Blocks” in the Context of Clinical Experience .
ten Cate O, Scheele F. Viewpoint: Competency-based postgraduate training: Can we bridge the gap between theory and clinical practice? Academic Medicine. Jun 2007;82(6):

7. EPA
Entrustment refers to the ability to effectively perform a professional activity unsupervised (GME) or without direct supervision (UME)
Brings trust and supervision into assessment which are intuitive for faculty working with trainees
Entrustment decisions allow inference about a learner’s competence

8. Entrustment is Based On:
Ability or level of KSA
Conscientiousness
Telling the truth – absence of deception (truthfulness)
Knowing one’s limits (discernment)
Kennedy et al., Acad Med. 2008;83(10 Suppl):S89–92

9. EPA
Professional is a modifier of activities that refers specifically to:
Area of practice (e.g. specialty)
Scope of practice (e.g. profession)
Point in an education-training-practice continuum (e.g. UME-to-GME or GME-to-practice transitions)

10. EPA
The Activities:
Represent the essential work that defines a discipline (in aggregate)
Lead to a recognized outcome
Should be independently executable within a given time frame
Are observable and measurable units of work in both process and outcome
Require integration of critical competencies and milestones

11. Identifying EPAs for a Specialty
What does a general pediatrician do (or should she do) in everyday practice?
Facilitate handovers to another health care provider
Care for the well newborn
Lead and work within interprofessional health care teams

12. EPA Worksheet ___ Patient Care Step 1. EPA Title
Step 2. Description of the activity
Brief overview and list of functions
Step 3. Map to Competency Domains
___ Patient Care
___ Medical Knowledge
___ Practice-Based learning & Improvement
___ Interpersonal & Communication Skills
___ Professionalism
___ Systems-Based Practice
___ Personal & Professional Development
Step 4. Map to Critical Competencies
Step 5. Curriculum
Step 6. Entrustment Decisions
Modified from the work of ten Cate

13. Step 1: EPA Title
Facilitate handovers to another health care provider either within or across settings

14. Step 2: EPA Description/Rationale
Healthcare system has increased in complexity
Commensurate increase in number and complexity of handovers
Within settings (e.g., hospital ER to inpatient)
Between settings (e.g., home to hospital)
Continuity of healthcare providers has decreased

15. Step 2: EPA Functions Information provider: Information Receiver
Communicate bi-directionally with patients, families and other caregivers re: plans and expectations across the transition
Navigate the information system (avoid errors of omission)
Communicate situation awareness, illness severity, action planning and contingency planning to other health care providers
Information Receiver
Elicit clarifying information
Provide feedback to the information provider on any errors of omission or inaccurate information transmitted

16. Step 3: Judicious Mapping to Competency Domains
X Patient Care
Medical Knowledge
X Practice-Based Learning and Improvement
X Interpersonal & Communication Skills
Professionalism
Systems-Based Practice
Personal & Professional Development

17. Step 4: Judicious Mapping to Competencies Within Domains
Patient Care
Organize and prioritize responsibilities to provide patient care that is safe, effective and efficient
Provide transfer of care that insures seamless transitions
Interpersonal and Communication Skills
Communicate effectively with physicians, other health professionals and health related agencies
Maintain comprehensive, timely and legible medical record
Practice-based Learning and Improvement
Incorporate formative evaluation feedback into daily practice
Use information technology to optimize learning and care delivery
Note the importance of Judicious mapping AND the need to map across all competencies in the aggregate. Note the current struggle with this concept-are some competencies not amenable to assessment through an EPA framework?

18. C2 DOC C3 C1 DOC EPA C4 C2 DOC C5 M1 Mx M1 Mx M1 Mx M1 Mx M1 Mx M1 Mx
EPA: Entrustable Professional Activity
DOC: Domain of Competence C: Competency M: Milestone M1 C5 Mx

19. Step 4A: Painting Pictures of Learners to Inform Assessment
Use table columns to construct expected behaviors of learners at different levels

20. EPA: Facilitate Handovers Milestone Series for a Given Competency
Domains & Competencies
Milestone 1
Milestone 2
Milestone 3 …etc PC
Organize and prioritize
Provide transfer of care
Novice
behaviors
Advanced
beginner
Competent behaviors
ICS
- Communicate with physicians, other health professionals and agencies
-Maintain medical records
PBLI
-Incorporate feedback into practice
-Use IT to optimize care
So if you judiciously map EPAs to domains of competence, competencies and milestones you can create a matrix in which you list the competencies in the left hand column and the series of milestones attached to that competency in the adjacent row. And then just like we do with the Denver Developmental Screening Test we can drop a perpendicular down one of the Milestone levels and get a snapshot of all the behavioral descriptors of a learner engaged in that professional activity at that performance level. We are then rolling those descriptors into clinical vignettes for each performance level and are testing them as assessment tools. There has been some work in this area that has been promising so we are hopeful…
The key word here is JUDICIOUS! Effective, efficient and meaningful assessment needs to drive the mapping process. What competencies are critical to making an entrustment decision? Remember we have 19 EPAs. We want to ensure that somewhere in the maps of the 19 all of the 51 competencies are covered. Not every competency needs or should be assessed in every professional activity. It doesn’t make sense to do this.
Is it messy? YES
Is it meaningful? YES
It is clearly better to sit with a learner and review a vignette and what behaviors match theirs and what behaviors they need to work on and improve that it is to give someone a “meets expectations” or a 3 on a 5 point Likert scale!

21. M1 Mx M1
Narrative description of an early (novice) learner Mx M1 Mx M1 Mx
Narrative description of a learner at “x” level M1 Mx M1 Mx

22. Step 4B: Creating Vignettes from the Expected Behaviors
Use expected behaviors to develop vignettes that could be used for faculty development and vignette matching
Ideal state: Create video scripts and videos from the vignettes

23. Step 5: Develop Curriculum
List specific knowledge, skills and attitudes that are needed
Decide the how, when, why and who for teaching and assessing the EPA in your program

24. Step 6: Entrustment decisions
Who will make the entrustment decision
Individual?
Group?
Learner Input?
On what basis
Required Assessments?
Contexts?
How many observations?

25. Small Group Exercise Vignette Ranking
Read the five vignettes individually and rank them from lowest (level 1) to highest (level 5) of performance
Discuss at your tables and come up with consensus ranking
Decide as a group which level would equate with an entrustment decision
Large group ranking and entrustment discussion

26. Individual and Small Group Exercise: Video Vignette Matching
Watch the first video and match the learner providing the handover communication to one of the five vignettes
Discuss at your table and come to a consensus on which vignette matches the video
What feedback would you give to the learners to help move them to the next level of performance?
Large group report out

27. EPAs across the continuum
UME
UME
GME
Practice

28. Entry into Clerkship EPAs (UCSF)
1) Gather information from a medically stable patient with a common chief complaint 2) Integrate information gathered about the patient to construct a reasoned and prioritized differential diagnosis and preliminary plan for common chief complaints 3) Communicate information relevant to patient’s care with other members of the health care team 4) Share information about the patient’s care, including diagnosis and management plan, with a patient in no significant physical or emotional distress 5) Provide the health care team with resources to improve an individual patient’s care or collective patient care

29. Core EPAs for Entering Residency
Gather a history and perform a physical examination
Prioritize a differential diagnosis following a clinical encounter
Recommend and interpret common diagnostic and screening tests
Enter and discuss orders/prescriptions
Document a clinical encounter in the patient record
Provide an oral presentation of a clinical encounter
Form clinical questions and retrieve evidence to advance patient care

30. Core EPAs for Entering Residency
Give or receive a patient handover to transition care responsibility
Collaborate as a member of an interprofessional team
Recognize a patient requiring urgent or emergent care, and initiate evaluation and management
Obtain informed consent for tests and/or procedures
Perform general procedures of a physician
Identify system failures and contribute to a culture of safety and improvement

31. List of General Pediatrics EPAs
1. Provide consultation to other health care providers caring for children
2. Provide recommended pediatric health screening
3. Care for the well newborn
4. Manage patients with acute, common diagnoses in an ambulatory, emergency, or inpatient setting.
5. Provide a medical home for well children of all ages. (Entrustment decisions for this EPA may require stratification by age group)
6. Provide a medical home for patients with complex, chronic, or special health care needs. (Entrustment decisions for this EPA may require stratification by age group)

32. List of General Pediatrics EPAs
Recognize, provide initial management and refer patients presenting with surgical problems
Facilitate the transition from pediatric to adult health care
Assess and manage patients with common behavior/mental health problems.
Resuscitate, initiate stabilization of the patient and then triage to align care with severity of illness (Entrustment decisions for this EPA may require stratification by two age groups: neonate and non-neonate).
Manage information from a variety of sources for both learning and application to patient care

33. List of General Pediatrics EPAs
12. Refer patients who require consultation
Contribute to the fiscally sound and ethical management of a practice (e.g. through billing, scheduling, coding, and record keeping practices)
Apply public health principles and quality improvement methods to improve care and safety for populations, communities, and systems
Lead an interprofessional health care team
Facilitate handovers to another healthcare provider either within or across settings
Demonstrate competence in performing the common procedures of the general pediatrician

34. Beyond Residency: EPAs for Subspecialties

35. EPAs across the continuum
Entering Clerkship EPA:
Communicate information relevant to patient’s care with other members of the health care team
Core EPA for Entering Residency:
Give or receive a patient handover to transition care responsibility
Pediatric and Subspecialty EPA:
Facilitate handovers to another healthcare provider either within or across settings

36. EPAs Across the Continuum
Entering Clerkship EPA:
Integrate information … to construct a reasoned and prioritized differential diagnosis and preliminary plan
Core EPA for Entering Residency:
Prioritize a differential diagnosis following a clinical encounter
Recommend and interpret common diagnostic and screening tests
Enter and discuss orders/prescriptions
Pediatric EPAs
Manage patients with acute, common diagnoses in an ambulatory, emergency, or inpatient setting
Subspecialty EPAs
Manage patients with acute endocrine disorders in ambulatory, emergency or inpatient settings

38. Milestones + EPAs Both Are Critical for Assessment
Competencies & Milestones: A Granular Approach (Telephoto)
Assess a level of performance (milestone) for a specific learner ability (competency)
Provide the substrate for diagnosis for learners with difficulty
EPAs: A Holistic Approach (Panoramic)
Integrate competencies within a clinical context
Assess clusters of behaviors required to carry out a professional activity
Map to competencies & milestones - vignette matching
So the bottom line is that Milestoens and EPAs are both critical for assessment.

39. Summary: Why EPAs? Make sense to faculty, trainees and the public
Make assessment more practical by clustering competencies and their milestones into meaningful professional activities
Add meaning to assessment by focusing on integration of competencies in the context of care delivery
Add “trust” to the assessment conversations
Align what we assess with what we do

40. Objectives
Develop a working knowledge of EPAs and their relationship to competencies and milestones
Apply that working knowledge to a specific example of a handover communication EPA
Make the connections between EPAs across the Education and Training Continuum

41. Questions/Reflections