Director, Competency-based Assessment Programs SCTC Meeting – May 23, 2012 Entrustable Professional Activities (EPAs): The Competencies and Milestones in Context Carol Carraccio, MD,MA Director, Competency-based Assessment Programs

Objectives Answer the question: What are EPAs & why do we need them when we have competencies and milestones? Explain the added value of EPAs to assessment Discuss example EPAs- their descriptions and step by step mapping to competencies and milestones Explain why identifying EPAs is a team sport

Setting the Stage 51 series of milestones that span the novice to expert continuum & contribute meaningfully to assessment by: Providing descriptions of behaviors that ground the more abstract language of competencies Informing us about learners at a granular level as we directly observe their behaviors Providing substrate for formative feedback and a learning roadmap BUT

Setting the Stage Need to integrate the competencies/milestones for a more holistic assessment of learners to answer the question “Is he a good doctor?” AND Cluster the milestones to make assessment more practical & situate it in a clinical context to make it more meaningful (Entrustable Professional Activities- EPAs)

Where Do We Go From Here? The Good News About Milestones Like the competencies they are context-free The Bad News About Milestones Translation: Current milestones span the continuum so no need to write new milestones for subspecialties but need to embed existing milestones into subspecialty EPAs

Context Matters The Milestones as “Building Blocks” in the Context of Clinical Experience: Introducing EPAs . As we said the milestones are helpful in understanding the language of the competencies and helping us to assess learners at a more granular level as they perform components of the competencies but there is still something missing- and that is the grounding of the competencies in a clinical context which helps us to assess learners in integrating competencies and milestones to provide actual care to patients. The missing link is the construct of entrustable professional activities or EPAs. ten Cate O, Scheele F. Viewpoint: Competency-based postgraduate training: Can we bridge the gap between theory and clinical practice? Academic Medicine. Jun 2007;82(6):542-547

Entrustable Professional Activities In aggregate- represent the essential professional work that defines a discipline Lead to a recognized outcome Are observable and measurable Require integration of competencies (KSA) across domains Map to competencies and their milestones

The Good Doctor: PUTTING IT ALL TOGETHER Domains of EPAs Competence Competencies Milestones EPAs DOMAINS OF COMPETENCE COMPETENCIES MILESTONES The Good Doctor: PUTTING IT ALL TOGETHER

The “Entrustable” in an EPA Ability to perform an activity at a desired level of performance without direct supervision Trust is (should be) based on directly observed, consistent performance over time We constantly, often subconsciously, make entrustment decisions based on needed level of supervision- need to formalize this process

Why Focus on Entrustment? Adds meaning to assessment “Do you trust this person to provide care for a patient referred with an acute cardiac problem without direct supervision?” Versus “Is this person competent in PBLI, SBP, etc?” 10

Summary: Why EPAs? Make sense to faculty, trainees, and the public Situate competencies and milestones in the clinical context in which we live Align what we assess with what we do Add meaning to assessment by focusing on integration of competencies in care delivery Make assessment more practical by clustering 51 series of milestones into meaningful professional activities 11

Identifying EPAs: Begin With the End in Mind What does a suspecialist do in everyday practice? Translates into the EPAs for subspecialty training

EPAs for Subspecialties Three Broad Categories Common across the generalist/subspecialist role Common across subspecialties Subspecialty-specific

EPAs Common to Generalists/Subspecialists Provide consultation to other health care providers caring for children Provide consultation using a variety of media (e.g. telephone, e-mail, webcast, video conferencing) Facilitate the transition from pediatric to adult health care Perform operational functions in a group practice setting Improve care for a population of patients Lead a health care team Facilitate handovers to another healthcare provider either within or across settings Expectations are higher for fellows than residents and higher for residents than students

EPAs Common Across Subspecialties Contribute to the scholarly work of the subspecialty Engage in meaningful MOC to fill knowledge/skills gaps and demonstrate deliberate practice Co-manage patients with generalists and other subspecialists Effectively perform the procedures key to the subspecialty

Subspecialty Specific EPAs Sub-categories Care for patients with acute ______ problems Provide a medical home for patients with chronic _____ problems Care for patients who require ______ (some unique non-procedural skill (e.g., management s/p transplantation) For subspecialties like cardiology who require additional training for EP consider an EPA on EP for which general cardiology fellows are not expected to achieve entrustment In the case of tiered subspecialties like cardiology, you may not expect entrustment for every EPA. You would not expect a general cardiology fellow to be entrusted to perform a cardiac cath without direct supervision but you may expect him to be able to know who to refer and how to interpret the data from a cath.

EPA Worksheet 1. EPA Title Improve care for a population of patients 2. Activity Description The 21st century health care professional needs to understand population health in order to optimize care. Populations can be defined by socio-demographics, disease states, and/or active practice patients to name a few. The functions required: Apply knowledge of population health Function in an interdependent health care team Collaborate with others to improve systems Recognize one’s professional responsibility to populations, communities and society at large Utilize technology (e.g. patient registries and databases) Demonstrate adaptability in developing and implementing improvement plans Utilize risk/benefit and cost/benefit analysis

EPA Worksheet (continued) 3. Judicious Mapping to Domains of Competence ___ Patient Care ___ Medical Knowledge _x_ Practice-Based Learning/Improvement ___ Interpersonal & Communication Skills _x_ Professionalism _x_ Systems-Based Practice _x_ Personal & Professional Development Modified from the work of ten Cate

EPA Worksheet (continued) 4. Judicious Mapping to Competencies Practice-based Learning and Improvement Systematically analyze practice using quality improvement methods, and implement changes with the goal of practice improvement Use information technology to optimize learning and care delivery Professionalism Develop a professional identify, including understanding, appreciation, and internalization of the professional role as it relates to patient, community, or specialty

EPA Worksheet (continued) 4. Judicious Mapping to Competencies Systems-based Practice Incorporate considerations of cost awareness and risk-benefit analysis Advocate for quality patient care and optimal patient care systems Know how to advocate for the promotion of health and the prevention of disease and injury in populations Personal & Professional Development Flexibility and maturity in adjusting to change with the capacity to alter one’s own behaviors

EPA Worksheet (continued) 5. List specific knowledge, skills and attitudes that are needed to execute the EPA well? Curriculum KSA specific to: Quality improvement methods Information technology Professional identity Cost awareness Risk-benefit analysis Quality care and care systems Advocacy Flexibility and maturity in adjusting to change

Next Steps for EPAs: Assessment Create a matrix (handout) Using milestones for each competency at each level of performance Synthesize behaviors across domains/competencies by level of performance Create clinical vignettes that describe the integration of behaviors across competencies at each level of performance Garner consensus about which level of performance correlates with “entrustment” of a learner and whether entrustment is required for that professional activity

Example Subspecialty EPA Worksheet 1. EPA Title Provide care to patients who present with acute renal problems 2. Activity Description The functions required: Apply in-depth knowledge of the field in implementing a focused diagnostic work-up and management plan Communicate with the patient and family and engage them to the extent possible in shared decision making Communicate with the referring physician and other team members involved in care Support the emotional response of patients and families to uncertain or fear inducing diagnoses, treatments, and/or prognoses An iterative process will be needed to sort functions that are more important to patients with chronic renal disease from those with acute renal problems such as helping to navigate the health care system. Likely procedural competence is so critical to subspecialty care that this will be identified as its own EPA so I did not include it as a function of the given EPA.

Example Subspecialty EPA Worksheet 3. Judicious Mapping to Domains of Competence _X__ Patient Care _X__ Medical Knowledge ___ Practice-Based Learning/Improvement _X__Interpersonal & Communication Skills ___ Professionalism __ Systems-Based Practice ___ Personal & Professional Development Modified from the work of ten Cate

Example Subspecialty EPA Worksheet 4. Judicious Mapping to Competencies Patient Care Make informed diagnostic & therapeutic decisions that result in optimal clinical judgment Develop & carry out management plans Medical Knowledge Demonstrate sufficient knowledge of the basic & clinically supportive sciences in the subspecialty

Example Subspecialty EPA Worksheet 4. Judicious Mapping to Competencies Interpersonal and Communication Skills Communicate effectively with patients, families and the public, as appropriate, across a broad range of socioeconomic and cultural backgrounds Communicate effectively with physicians, other health professionals, and health related agencies Demonstrate the insight and understanding into the emotion and human response to emotion that allow one to appropriately develop and manage human interactions

Example Subspecialty EPA Worksheet 5. List specific knowledge, skills and attitudes that are needed to execute the EPA well? Curriculum KSA specific to: Epidemiology, signs, symptoms, presenting features & up to date management of HUS, ATN… Interpreting diagnostic imaging & testing… Delivering bad news Basic elements of emotional intelligence …. Use your functions to help you inform your curriculum

Both are Critical to Meaningful Assessment Take Home Messages Milestones + EPAs: Both are Critical to Meaningful Assessment

Take Home Messages Milestones: A granular approach to assessment Address the learner at the level of a competency (e.g., gather essential information…) Allow drill down to smaller elements of behavior that provide the substance of formative feedback, remediation, and a learning roadmap EPAs: A holistic approach to assessment Integrate competencies within a clinical context Assess clusters of behaviors required to care for patients

Take Home Messages COLLABORATION IS KEY Identifying EPAs for Subspecialties: COLLABORATION IS KEY

Thank you!