1. Defining and Assessing Entrustable Professional Activities
Karen E. Hauer, MD
H. Carrie Chen, MD, MSEd
John Q. Young, MD, MPP
Patricia S. O’Sullivan, EdD
With thanks to Olle ten Cate, PhD
UCSF
June 5, 2013

2. Agenda 1-1:15 Welcome 1:15 Introduction to EPAs
1:45 individual writing of an EPA
2:05 overview of EPA development
2:35 small group activity, design an EPA (steps 1-4)
Break
3:15 small group reporting
3:45 overview of steps 5-7
4:00 second activity, evidence for an EPA (steps 5-7)
4:30 small group reporting and wrap up (all)

3. Workshop objectives
By the end of this workshop, participants will be able to:
Define an entrustable professional activity (EPA)
Develop an entrustable professional activity that can be used for assessment of medical trainees

4. What do you hope to learn today?

5. Current Assessment System
ACGME core competencies:
Patient Care
Medical Knowledge
Practice-based Learning and Improvement
Interpersonal and Communication Skills
Professionalism
Systems-based Practice 

6. Current Assessment System
Practice-based Learning and Improvement (Housestaff)  (Question 5 of 9 - Mandatory) Assimilates evidence from scientific studies or consultants related to their patients' health problems. Participates in organized transitions of care curriculum as specified at each site. Actively seeks out and acts on performance feedback from the supervising attending and other team members.
Most front line attendings don’t really know what PBLI means and certainly haven’t read the descriptor provided on the form
Insufficient Contact to Judge 
1 & 2 = Unsatisfactory 3 = Marginal    
Satisfactory/Meets Expectations 
7 & 8 = Excellent 9 = Outstanding  1 2 3 4 5 6 7 8
>> 9 <<

7. Current Assessment System
What are some limitations of current assessment systems?
Core competencies and sub-competencies: long checklists of behavioral descriptors
Relies on traditional but limited assessment methods – knowledge exams, ward evaluations
Not a holistic summative view of the trainee
When we consider the goal of application in the workplace, we start to better understand the challenge with competencies
…superficial and less reliable – did you wash hands? Did you ask an open ended question?

8. Competency
An observable ability of a health professional, integrating multiple components such as knowledge, skills, values and attitudes.
Let’s step back and define important terms in assessment. Form I just showed you listed a competency
The International CBME Collaborators, 2009

9. Competent
Possesses the required abilities in all domains in a certain context at a defined stage of medical education or practice.
a competent practitioner……
The International CBME Collaborators, 2009

10. Competence does the job
Competence entails more than the possession of knowledge, skills and attitudes; it requires you … to apply these [abilities] in the clinical environment to achieve optimal results.
does the job
Competence is the ultimate goal
ten Cate, Med Teach, 2010

11. Can you trust the learner to function independently?

12. How to Drive a Car Competency Competent Competence
Can accelerate and brake smoothly
Can approach an intersection and can turn left
Competent
Passes driver’s education classes
Passes driver’s exam to get the license
Competence
Drives safely on interstate or during bad weather, avoids accidents, no traffic tickets
Parents hand over the keys and walk away

13. What is the goal with assessment?
Integrated, longitudinal, learner-centered assessment system
Promote skill acquisition in multiple domains concurrently
Assess what learners actually do in practice
Be able to conclude: this is a trustworthy trainee
Although we all hear that our trainees are a pleasure to work with – we aspire to more. The goal would be …

14. Features of genuine competency based medical education
Outcome-based, not process-based: what is attained is key, not just what is done or taught
Integrates knowledge, skill, attitude
Time-independent: length of training for defined outcomes is not pre-set
Individualized: trainees and contexts vary
Workplace-learning based
Lifelong learning oriented
Not just the form we are using, not a checklist of competencies and milestones
we don’t promote lifelong learning
just because our residents are superb doesn’t mean we don’t need to assess them

15. Competence. What criteria would you use to select your doctor?
Passed all tests and exams?
Grades and scores?
Years of training?
Follows protocols and guidelines?
Trust that s/he will manage a case in the best possible way?

16. Entrustable Professional Activities (EPAs)

17. Entrustable Professional Activity (EPA)
Define important clinical activities
Link to competencies / milestones
Include professional judgment of competence by clinicians
Make “decisions of entrustment” for “entrustable” activities
Something changes for the learner….
Build collection of mastered EPAs (portfolio) to document full competence
EPAs together constitute the core of the profession

18. Definition of an Entrustable Professional Activity
A core unit of work, reflecting a responsibility that should only be entrusted upon someone with adequate competencies
More specific:
Part of essential professional work in a given context
Independently executable, within a time frame
Leads to recognized output of professional labor
Observable and measurable in process and outcome, leading to a conclusion (“well done” or “not well done”)
Must require sufficient, specific knowledge, skill and attitude, generally acquired through training
Should reflect competencies, important to be acquired
Usually confined to qualified personnel only

19. Competencies versus EPAs
person-descriptors
knowledge, skills,
attitudes, values
content expertise
collaboration ability
communication ability
management ability
professional attitude
scholarly approach
work-descriptors
essential parts of
professional practice
discharge patient
counsel patient
lead family meeting
design treatment plan
perform paracentesis
resuscitate if needed
Competencies
EPAs
Risk of checklist approach
142 milestones
PBLI on Evalue

20. Competencies and EPAs combined
Patient Care
Medical Knowledge
PBLI
Communi-cation
Professio-nalism
System-Based
Practice
Develop and implement a safe discharge plan for a patient from the acute care setting
xxx x xx
Lead a family meeting to discuss serious news with patient/family and other health providers

21. Exercise Complete Worksheet A on your own
List good examples of essential EPAs your trainees should be able to do independently at the end of training.
List bad examples of EPAs
Pair Share: share your examples with one person next to you.
JQY Leads

22. Recommended full EPA description
EPA Title (max 20 words, avoid skill and avoid adjectives)
Description of the activity (to serve universal clarity, include limitations)
Expected KSA (to serve trainee)
Link with competencies and predefined milestones (to embed within the existing framework)
Sources of information to determine progress (to serve observation and assessment)
Basis for formal entrustment decision (who will have a say in the decision -- signatures if formal and documented)
Post level-4 of entrustment (“unsupervised”) (what difference does it make for the trainee?)

23. Example from Pediatrics
One of the 16 draft EPAs developed by AAMC and ABP team for pediatric residency training:
EPA Title: Manage patients with acute, common single system diagnoses in an ambulatory, emergency, or inpatient setting
Description: Merged with expected KSA and clarified in descriptive vignettes

24. Example EPA cont’d Expected KSA:
Gathering info thru hx, PE, and initial labs
Sound clinical reasoning driving development of DDx to allow proper diagnostic testing and initial therapy
Knowledge of evidence related to primary problem
Application of evidence to management plan
Patient and family-centered care with bidirectional communications
Documentation of plan and reasoning that is transparent to other members of health care team

25. Example EPA cont’d Link with competencies/milestones: Patient care
Perform complete and accurate PE
Make informed diagnostic and therapeutic decisions that result in optimal clinical judgment
Develop and carry out management plan
Medical knowledge
Interpersonal and communication skills
Communicate effectively with patients, families, and the public as appropriate, across a broad range of socioeconomic and cultural backgrounds
Maintain comprehensive, timely, and legible medical records

26. PE Sub-competency
Performs essentially the same rote head-to-toe physical examination of the patient regardless of presenting complaint; does not use diagnostic hypotheses from the history to anticipate or look for specific positive or negative findings on physical examination.
With a broad list of diagnostic hypotheses after the history, uses a head-to-toe approach to the physical examination to anticipate and look for a myriad of potential positive and negative physical examination findings for multiple diagnostic considerations. This approach can lead to failure to identify pertinent and important physical findings that are present, misinterpretation of physical findings, and attribution of importance and meaning to irrelevant findings.
Uses a narrow list of diagnostic hypotheses generated through the history to anticipate and look for specific positive or negative physical examination findings of only the most relevant diagnostic considerations; open to new diagnostic possibilities in the process of performing a survey physical examination to elicit unexpected abnormalities but may dismiss these as unimportant when it is difficult to integrate these findings into the working differential diagnosis.
Uses a narrow list of diagnostic hypotheses generated through the history as well as through extensive clinical experience to anticipate and look for key specific physical examination findings that will discriminate between competing similar diagnoses; uses surprises that result from a survey physical examination to rethink and retest diagnostic hypotheses; actively looks for physical exam findings that disconfirm the working diagnosis or rule in or out rare but high-risk alternative diagnoses.

27. Description
Behaviors of early, more advanced, competent, proficient, and expert learner
Expected elements from milestones
Vignette – 2 yo with wheezing, resp distress
Reassurance provided during hx
Has FB aspiration on differential so focuses on differential BS in addition to wheezing and WOB
Presents focused hx/PE with reasoned assess/plan consistent with family’s wishes and health literacy, including SW referral for loss of insurance

28. Small Group Activity: Design an EPA
Work in small groups
Use resources provided
Internal medicine milestones
SOM milestones
Develop an EPA for
Internal medicine OR
Medical students

29. Small Group Debrief

30. When is “competence” reached? When you trust the trainee
When a professional activity is mastered
on a threshold level
that permits unsupervised practice
and full entrustment
It happens all the time: when trainees work without direct supervision
Ask for examples

31. Level of supervision Level 1: not allowed to practice the EPA
Level 2: practice with full supervision
Level 3: practice with supervision on demand
Level 4: “unsupervised” practice allowed
Level 5: supervision task may be given
Distinguish from billing requirements

32. Recommended full EPA description
EPA Title (max 20 words, avoid skill and avoid adjectives)
Description of the activity (to serve universal clarity, include limitations)
Expected KSA (to serve trainee)
Link with competencies and predefined milestones (to embed within the existing framework)
Sources of information to determine progress (to serve observation and assessment)
Basis for formal entrustment decision (who will have a say in the decision -- signatures if formal and documented)
Post level-4 of entrustment (“unsupervised”) (what difference does it make for the trainee?)

33. Dreyfus and Dreyfus Model
Novice – Don’t know what they don’t know
Advanced Beginner – Know what they don’t know
Competent – Able to perform the tasks and roles of the discipline – restricted breath and depth
Proficient – In depth knowledge concerning the discipline – often rule based – know what they know
Expert – Expert thrives with situations that break the rules – who the proficient practitioners go to for help

34. Development of Competence
expert
proficient
competent
advanced
Dreyfus and Dreyfus
Novice – follows rules, isolated facts, no context (if wheezing, albuterol)
Advanced beginner – begins to apply rules to other situations, synthesis/integration of info, decisions still rule based (sometimes so tight no wheezing, quiet breath sounds, albuterol)
Competent – organizing principles, sorting of relevant/irrelevant info, active responsible decision making (wheezing knows associated factors influence liklihood of asthma, has decision tree to allocate probabilities)
Proficient – intuitive/holistic diagnosis, based on real-world experience (this is a foreign body aspiration, applies rules to what to do next)
Expert – pattern recognition in diagnosis and plan, no decomposition into discrete elements to solve problem or make decisions (this is xx, we need to do yy)
Experienced non-experts vs true expertise
novice
training
deliberate professional practice
Dreyfus & Dreyfus, 1986 34

35. Sample Competency Curve
training
deliberate professional practice
EPA1
EPA4
EPA2
EPA3
EPA5
Competence
Threshold
Justified entrustment decisions
Graduating residents can graduate without reaching standard of practicing each EPA without supervision
Allows for learning after residency
Flexibility
Variation within trainees – not all EPAs reach competence threshold levels at same time
Variation among trainees – each with own speed/curve
Variation in context – ops, site, patients, culture 35

36. Small Group Activity: Choosing Evidence
What evidence will inform the EPA that you developed

37. Small Group Reporting

38. Wrap Up