1. Entrustable Professional Activities Implementations Moderated by Abby A. Kahaleh, BPharm, MS, PhD, MPH
Amy to start
September 12, 2017

2. Amy L. Pittenger, Pharm.D., M.S., Ph.D.
Report of the Academic Affairs Standing Committee: Entrustable Professional Activities Implementation Roadmap
Amy L. Pittenger, Pharm.D., M.S., Ph.D.
Chair, Academic Affairs Standing Committee

3. Committee Members Amy L. Pittenger, PharmD, PhD, Chair (Minnesota)
Debra A. Copeland, PharmD (Northeastern)
Matthew M. Lacroix, PharmD (New England)
Quamrun N. Masuda, PhD, RPh (VCU)
Peter Mbi, PharmD, PhD (Maryland)
Melissa S. Medina, EdD (Oklahoma)
Susan M. Miller, PharmD (UNC-Chapel Hill)
Scott K. Stolte, PharmD (Wilkes)
Cecilia M. Plaza, PharmD, PhD (AACP)

4. Charges
1) compiling comments and input from a broad range of stakeholders regarding the draft Entrustable Professional Activities (EPAs) from the Committee (including comments from District meetings) and complete final edits to the document for submission to the November 2017 AACP Board of Directors meeting;
2) develop potential uses and applications of EPA statements in pharmacy education; and
3) create a roadmap for implementation of EPAs across the member schools and colleges. Include in the plan a path to inform internal and external stakeholders regarding the EPAs and educational opportunities that are presented. Identify components of the plan that could be implemented starting fall 2017.

5. Final Set of Core EPAs
Published in AJPE Issue 1 for 2017 along with a commentary

6. Purpose of the report
Identify linkages across the EPA statements, Center for the Advancement of Pharmacy Education 2013 Educational Outcomes (CAPE 2013) and the Joint Commission of Pharmacy Practitioners’ Pharmacist Patient Care Process (PPCP);
Provide ways EPA statements can be used to communicate core skills that are part of the entry-level pharmacist identity;
Suggest a potential roadmap for AACP members on how to implement EPA statements.

8. Recommendations
Recommendation 1: AACP should work with APhA to link core EPA Statements for New Pharmacy Graduates into the APhA Career Pathways Program to better communicate how EPAs apply to various career paths.
Recommendation 2: AACP should encourage other organizations to use the series of 3 Academic Affairs Standing Committee Reports (including this report) to develop EPA statements for post-graduate specialized and advanced practice.1, 4
Recommendation 3: AACP should incorporate the Core EPA Statements for New Pharmacy Graduates into Priorities 1 and 2 in the AACP Strategic Plan

9. Recommendations
Recommendation 4: AACP should work with the Experiential Education Section and the AACP Master Preceptors to implement preceptor development related to the core EPAs (eg, webinar available to preceptors at any school)
Recommendation 5: AACP should work with other appropriate groups within the Association, identified through a peer review process, to develop on demand webinars for faculty, preceptors, and students on the core EPAs.
Recommendation 6: AACP should have an Institute to engage schools of pharmacy in developing their road map to core EPA implementation

10. Suggestion
Suggestion: Colleges/schools should identify which EPAs are addressed in each course and/or rotation syllabus using Appendix 1.

11. Roadmap Identify the team Map curriculum to EPAs Identify Milestones
Use existing curricular assessment activities
Leveling and Calibrating Each EPA Core Statement
Determine adequate progress
Pilot assessment with Students
Faculty training
Preceptor training
Student training

12. Using EPA for Communication
Matthew Lacroix, PharmD, MS, BCPS
Coordinator, for Assessment and Accreditation,
University of Rhode Island

13. Using EPA for Communication
EPA are regardless of setting
Unifying feature for the profession
Identity messaging
Informing future and current students
Engaging current pharmacist
Identifying role in healthcare team
Enlighten the public at large

14. Using EPA for Communication
Stakeholder agreement is essential to this process
Helps the academy train “Practice-Ready”
Illustrates the role in the healthcare team

15. Using EPA for Communication
Used to update the publically perceived role of a pharmacist
Widening the view from product tied dispensing to a medication focused cognitive skills provider.
Interprofessional team roles and responsibilities
Patient understanding of what the pharmacist can do
Inform policy makers of skills all pharmacists should have to modify pharmacy law and regulation

16. Using EPA for Communication
Internally inform the profession
Operationalize competency statements
Mapable to foundational science as well as clinical science
Unification across the pharmacy disciplines
Adjust self-directed learning to current best practices
Maintenance of competency, CPD
Develop career paths
Preceptors
Help them understand the training that is required within the curriculum
Provide feedback to assessments that measure competency prior to APPE

17. Using EPA for Communication
If accepted, can unify vision for pharmacy advocacy
Cohesive definition of “at graduate” skill set
Allows for building stronger post graduate training competencies
Allows for more focused CPD programming
Focused certificate programs for specialized skills

18. Using EPA for Communication
What a pharmacy graduate is capable of upon graduation
Help faculty, staff, preceptors and students how they are being assessed and where possible remediate skills for “practice ready”
Ultimately all done to show not tell how pharmacists impact patient care

19. Training on EPA through Integrated Simulated Learning
Quamrun Masuda, PhD, RPh
Associate Professor (Pharmaceutics)/Asstt. Dir., Center for Compounding Practice & Research
Virginia Commonwealth University
School of Pharmacy

20. Basic Science in PharmD Curriculum
As a pharmaceutical sciences faculty I ask myself:
Where are we today?
Where do we want to go tomorrow?
How do we go there?

21. Current Curriculum Status
Subject-based traditional curriculum
Basic science knowledge is loaded up front
Teaching happens in silos, not knowing what others are teaching
No link between textbook knowledge and practice

22. Where We Are Heading? Curriculum Change
We need to design integrated curriculum
Relevant concepts from foundational sciences are to be brought together in a coordinated way
Provide the big picture in disease management
Make teaching on disease state or organ systems with clinical orientation at early level (P1)
Make connection between basic & clinical sciences with applied examples at all levels (P1-P4)

23. EPA Demonstration in Diabetes Management
EPA in Diabetes management is demonstrated by graduates with minimum or no supervision.
This goal is achieved through phases of learning.
The flowchart here summarizes the integrated learning model.
The basic sciences knowledge are intermeshed with clinical sciences in phases of simulation, and experiential education.
This illustration shows role of BS faculty in facilitating skills development in diabetes management.

24. Content Integration
Let’s take an example of managing diabetes, as shown in the flowchart.
The following learning process is happening before managing a diabetic patient:
Metabolic process and conversion of carbohydrate, fat, and protein to glucose (Biomedical Sciences at P1 level)
Secretion of insulin to store glucose in the body and use it when is needed (Biomedical Sciences at P1 level)
Pathophysiology of pancreatic cells, both α & β cells of Langerhans (Biomedical Sciences at P1 level)

25. Content Integration (contd.)
Patient assessment: signs and symptoms of Type 2 diabetes (Clinical Science at P1 and P2 level)
Lab values and markers of hyperglycemia and related markers (Biomedical & Clinical Sciences at P1 & P2 level)
Does the lab values of the markers matches with the physical assessment? (Clinical Sciences at p2 level)

26. Content Integration (contd.)
Think about pharmacotherapy in coordination with:
Medicinal chemistry of antidiabetic drugs, its QSAR (Med Chem at P2)
Design of conventional and modified release dosage forms to synchronize circadian rhythm of hyperglycemia and drug release (Pharmaceutics/Biopharmaceutics at P1 & P2)
Pharmacokinetics, and pharmacodynamics of antidiabetic drugs (Pharm Sci. at P2)
Patient counseling: (OSCE simulation facilitated by Pharm Sci & Clinical faculty, & IPPE Preceptors at P1 & P3 year, & APPE preceptors at P4 year)
How to take
How to store
What signs and symptoms to expect
When to contact pharmacist or physicians
Health & wellness education on lifestyle (Clinical, biomedical sciences)
Vaccinations (Clinical and Pharmaceutical Sciences at P1 –P3 yr, APPE preceptors at P4 year)
Monitoring vision, foot care, cardiac & renal functions and guidelines of diabetes management (Clinical & Biomedical Sci. at P2-P3 year, APPE preceptors at P4 year)

27. Foundational Knowledge Integration in Diabetes Management
Pharmaceutical Sciences: Pharmaceutics/Biopharmaceutics
Route of Delivery:
Oral, Parenteral
Dosage form: IR, ER, IM, IV, Autoinjectors
Route based bioavailability
Adverse effects, Interaction
Biomedical Sciences: Physiology, Pathology, & Pathophysiology of Pancreatic cells
Physiology of pancreas
Pathology of hyperglycemia
Rationale for therapy & treatment
Clinical & Pharm Sci. Sciences: Medication Dispensing, Distribution, & Administration
Medication adherence
Proper use of insulin
Immediate released or extended released products
Disease-state Monitoring vision, foot care, Cardiac & renal function

28. Diabetic Complexity: Monitoring Multiple Organ Failure
Diabetes Management: Monitoring Peripheral Vascular Disease & Congestive Heart Failure
Foundational
Knowledge 1
Knowledge 2
Knowledge 3
Knowledge 4
Cardiac
Diabetic Complexity: Monitoring Multiple Organ Failure
Renal
Knowledge 5
Knowledge 6
Knowledge 7
Knowledge 8
Neurology

29. Implementation Steps Debra Copeland, BS, PharmD, RPh
Director, Office of Experiential Education
Associate Clinical Professor, Northeastern University

30. Implementation Steps Identify your team Map Curriculum to EPAs
Identify Milestones
Check if existing assessment can measure Milestones
Leveling and Calibrating

31. Implementation Steps Determine adequate progress
Systematically pilot assessment
Faculty education and training
Preceptor education and training
Student education and training
Identify your “level of entrustment” rubric

32. Step 1: Identify your team
Create Ad hoc committee
from Curriculum and/or Assessment Committee(s)
Led by administrator
Team member inclusion:
Didactic education faculty
Experiential education faculty
“Minute Readers”
e.g., preceptors, residency program directors, etc.
Identify “level of entrustability” rubric

33. Rubric: Level of Entrustability
Will be used to measure EPA accomplishment
Many exist in the literature
Haines ST. Table 1 in AJPE 2017; 81(1) Article 18
Pittenger AL. Table 2 in AJPE 2017; 81(5) Article S4
Ten Cate O. Nuts & Bolts. JGME 2013; 5(1): 157.

34. Describing Entrustable Professional Activities is Merely a First Step—Table 1
Haines ST. AJPE 2017; 81(1) Article 18

35. 2016-2017 Report: EPA Implementation Roadmap—Table 2
Pittenger AL. Table 2 in AJPE 2017; 81(5) Article S4http://

36. Nuts and Bolts of Entrustable Professional Activities
Level 1: Observation but no execution, even with direct supervision Level 2: Execution with direct, proactive supervision Level 3: Execution with reactive supervision (i.e., on request and quickly available) Level 4: Supervision at a distance and/or post hoc Level 5: Supervision provided by the trainee to a more junior colleagues
Ten Cate O. Nuts & Bolts. JGME 2013; 5(1): 157.

37. Step 2: Map curriculum to EPAs
Review curriculum to determine where each EPA* is addressed
Identify
what program year and course addresses each EPA
If any EPA is not addressed
If and how the level of entrustability advances across the curriculum

38. Step 2a*: Map to existing program EPAs
Compare program EPAs to AACP Core EPAs
Create a cross-walk map
Noting similarities and differences
Based on cross-walk
Adopt Core EPAs to replace existing program EPAs; OR
Retain existing program EPAs (sharing cross-walk map during self-study); OR
Use Core EPAs AND retain selective existing EPAs

39. Step 3: Identify Milestones for EPAs
Milestone = a time frame (aka checkpoint or cut point) when an EPA skill is measured to ensure students are progressing and developing as expected
Programmatic specific (based on curriculum)
Examples:
N=3: Before 1st IPPE, Before 1st APPE, Before graduation
N=4: end of P1, P2, P3 and P4 years

40. Step 4: Assessments for Milestones
Goal: revise/prune existing assessments to measure milestones
Start with existing assessments
Modify, replace or add assessments as needed
When and where can assessments be used
Example:
OSCE in skills lab could be modified to document milestone progression

41. Step 5: Leveling and Calibrating
After number and placement of milestones…
Leveling = determines what the expected level of entrustment is for each EPA at each milestone
Example of how do you leveling
Gather all faculty who teach at milestone
Ask them to identify level of entrustment (via your rubric)

42. Step 5: Leveling and Calibrating
After leveling…
Calibrating = determines if the level of entrustment set is for each milestone was accurate
Purpose of this step? Allows the faculty to determine if the levels set, for that milestone, is/are “just right”

43. Step 5: Leveling and Calibrating
Example of how do you calibrating:
Evaluate students at a given milestone point (e.g., end P1 year) on the level prescribed (level 2) to see if the expectation for the level matches the students’ abilities
Complicated step…therefore,
Include Evaluation & Assessment folks on your campus
EPA-focused Institute participation may assist in this process

44. Step 6: Determine “adequate progress”
Define number or percentage of EPAs, meeting the level of entrustability, for a given milestone that allows the student to progress to the next stage of the program
Determine what will halt progression, trigger remediation, require formative feedback
At graduation all students should be able perform each EPA at a level not requiring direct supervision or higher

45. Step 7: Pilot assessments with students
Adjust…adjust….adjust
Phase implementation
Pilot group of educators…pilot number of EPAs
AVOID full implementation at once

46. Step 8: Training - Faculty
Following pilot testing…
Educate program faculty (didactic and experiential) on:
Core EPAs
Milestone markers
Rubric
Consider expanding pilot test information for all faculty
Article offers examples for implementation

47. Step 9: Training - Preceptors
Educate program preceptors on:
Core EPAs
Milestone markers
Rubric
EPAs can serve to
Standardize preceptor development programming
Standardize a national evaluation tool

48. Step 10: Training - Students
Follows faculty & preceptor education
Educate program students on:
Core EPAs
Milestone markers
Rubric
Formative & Summative feedback

49. HOW EPA STATEMENTS CAN HELP STANDARDIZE APPE EVALUATION
Peter Mbi, Pharm.D., Ph.D.
AACP Master Preceptor

50. CORE EPA REALTED PRACTICE /AND YEAR
Core AACP-EPA General Overview: GRADING/ASSESSMENT SCALE
FIVE (5) LEVELS: Preceptor use to assess/grade student on EPA proficiency, rotation type and year.
Level 1
Level 2
Level 3
Level 4
Level 5
Level 5 –(Alternate) for exceptional student

51. LEVEL 5 ALTERNATE
LEVEL 5
LEVEL 4
LEVEL 3
LEVEL 2
LEVEL 1
Familiar
Student demonstrates CORE EPA skills to less experienced learner.
Student takes initiative to identify and solve problems related to CORE EPA practice
Student’s Performance is above graduate level
 Familiar
Student Consistently demonstrates accurate performance of CORE EPA
Student can identify and solve problems related to CORE EPA
Student is Familiarity with CORE EPA developed through practice.
Student is Mostly consistent in identifying or solving problems related to CORE EPA
Student begins or Initiates CORE EPA skills/practice
Student is Not able to consistently perform CORE EPA skills.
Student is barely able or Rarely able to identify or solve problems related to CORE EPA
Student understand his/her deficiency in CORE EPA
Student makes efforts or attempts to become competent in CORE EPA
Student does not understand how to approach CORE EPA
 Capability
Student is consistently confident and able to complete CORE EPA skills.
Student demonstrates excellent depth and breadth of understanding of key content CORE EPA and knowledge and applies consistently
Student is able to independently complete CORE EPA skills
Student has the ability to adequately complete CORE EPA.
Student unable to perform CORE EPA independently
Student needs quidance
Student Sometimes is able to complete CORE EPA
Student occasionally Sporadically makes effort, but rarely able to perform CORE EPA skill
Student Lacks ability to independently complete CORE EPA skills
 Guidance
Student hardly requires intervention.
Student performs CORE EPA skills independently.
Student incorporates feedback
Student requires minimal quidance or Supervision
Student sometimes seek feedback from preceptor with specific questions CORE EPA performance
Student needs Supervision to master major CORE EPA concepts
Student requires less Supervision and feedback to complete CORE EPA skills
Student require or needs Constant guidance or supervision and feedback to complete CORE EPA skills
Student requires major guidance or supervision to complete CORE EPA skills
 Errors
Student hardly makes minor mistakes/errors
Student occasionally makes minor mistakes/errors
Student makes No major errors or mistakes
Student makes Minor mistakes
Student is Able to self-correct
Student makes less Major mistakes when attempting CORE EPA
Student is not able to self-correct most errors.
Student makes Major errors or mistakes when attempting to complete CORE EPA skills.
Student is not able to self-correct
Student make Major errors or mistakes when attempting to complete CORE EPA skills.
Student puts Patient safety at risk

52. How preceptors can apply scores
Evaluating/Grading Student Performance:
Preceptor selects rating in each level for each EPA and add comments. 
Preceptors are encouraged to provide ongoing or daily formative feedback to students to improve performance of EPA skills, and to guide EPA activities throughout the rotation
Comments from preceptors are strongly encouraged
Midpoint evaluation strongly encouraged. This can help to track student progress in EPA, provides formative feedback for students to improve performance, and guides activities for the remainder of rotation. 
If the student’s overall performance at midpoint is deficient, preceptors are encouraged to contact the student school or college.

53. Performance Outcomes Criteria
Preceptors are encouraged to evaluate the student at both the mid-point and at the end of the rotation, using EPA competency levels and descriptors.
Each performance item on the assessment domain, should or can be rated using the competency levels of 1, 2, 3, 4, 5 or 5 Alternate.
Preceptors can select the competency level that best describe the student’s performance at the point of assessment.

54. Example of EPA Domain & Core Statements
1. PATIENT PROVIDER - Collect information to identify a patient’s medication-related problems and health related needs - Analyze information to determine the effects of medication therapy, identify medication-related problems, and prioritize health-related needs - Establish patient-centered goals and create a care plan for a patient in collaboration with the patient, caregiver(s), and other health professionals that is evidence-based and cost-effective. - Implement a care plan in collaboration with the patient, caregivers, and other health professionals. - Follow-up and monitor a care plan.

55. An example of how a preceptor can use an EPA domain to evaluate a student.
The next slides is an example:
Preceptors can use level 1 through 5 (recorded under “no score selected”)to evaluate a student during midpoint and final.
Comments from preceptors are strongly encouraged

56. Domain 1: Analyze information to determine the effects of medication therapy, identify medication related problems, and prioritize health-related needs.
Skill – Prospective EPA skill or tasks
Asses a patient’s signs and symptoms to determine whether the patient can be treated within the scope of practice or require a referral.
Measure an adult patient’s vital signs and interpret the results (e.g. body temperature, pulse rate, respiration rate, and blood pressure)
Recommend laboratory tests and interpret the results.
Identify drug interactions.
Perform a comprehensive medication review (CMR) for a patient.
Assess a patient’s health literacy using a validated screening tool.
Compile a prioritized health-related problem list for a patient.
Evaluate an existing drug therapy regimen.
NO SCORE
SELECTED
NO SCORE SELECTED
Question Comments (Midpoint): Question Comments (Final):

58. REFERENCES
Medina MS, Plaza CM, Stowe CD, et al. Center for Advancement of Pharmacy Education 2013 Educational Outcomes. Am J Pharm Educ. 2013;77(8):Article 162
Joint Commission of Pharmacy Practitioners. Pharmacists’ Patient Care Process.
Haines ST, Pittenger AL, Stolte SK, Plaza CM, Gleason BL, Kantorovich A, McCollum M, Trujillo JM, Copeland DA, Lacroix MM, Masuda MM, Mbi P, Medina MS. Core entrustable professional activities for new pharmacy graduates. Am J Pharm Educ. 2017; 81: article S2.

59. An example of how a preceptor can use an EPA domain to evaluate a student.
The next slides is an example:
Preceptors can use level 1 through 5 (recorded under “no score selected”)to evaluate a student during midpoint and final.
Comments from preceptors are strongly encouraged

60. Mapping EPA to CAPE outcomes and Pharmacists’ Patient Care Process
Melissa Medina, Ed.D.
Mapping EPA to
CAPE outcomes and
Pharmacists’ Patient Care Process

61. Linkage Across EPA Statements
CAPE 2013 & Pharmacist Patient Care Process formed the foundation of EPA statements
Let’s review the EPA Statements, CAPE 2013, & PPCP…..

62. EPAs – 6 Domains Self-developer Pt Provider
Interprofess-ional Team Member
Population Health Promoter
Information Master
Practice Manager
Self-developer

63. CAPE 2013 Domains & Subdomains
1. Foundational knowledge
1.1 Learner (Learner)
2. Essentials for practice
& care
2.1 Patient-centered care (Caregiver)
2.2 Medication use systems management (Manager)
2.3 Health and wellness (Promoter)
2.4 Population-based care (Provider)
Approach to practice
3.1 Problem solving (Problem-solver)
3.2 Educator (Educator)
3.3 Patient advocacy (Advocate)
3.4 Interprofessional collaboration (Collaborator)
3.5 Cultural sensitivity (Includer)
3.6 Communication (Communicator)
4. Personal &
professional development
4.1 Self-awareness (Self-assessor)
4.2 Leadership (Leader)
4.3 Innovation and Entrepreneurship (Innovator)
4.4 Professionalism (Professional)
A one word descriptor in parenthesis is provided for each subdomain that illustrates the main construct and can be used to concisely articulate the outcome.

64. Pharmacists’ Patient Care Process

65. Linkages
EPAs operationalize the CAPE 2013 into discrete measurable tasks
Demonstration of EPAs requires proficiency in multiple competencies simultaneously
EPAs don’t represent new concepts but instead translate those concepts into measurable tasks
A map was developed to show linkages of EPAs to CAPE & PPCP

66. Domain 1: Patient Provider
EPA
Collect info on pt. related problems & health needs
Collect
2.1, 2.2, 2.3,
3.1, 3.5, 2.6
Analyze Info
Assess
1.1, 2.1,
3.1, 3.5
Est. pt. goals & create care plan
Plan
1.1, 2.1, 2.2,
3.3, 3.5
Implement pt. care plan
Implement
2.1, 2.2,
3.4, 3.6
Follow-up & monitor care plan
Follow-up: monitor & eval
Domain 1: Patient Provider
C=EPA subdomain; circle = PPCP

67. Domain 2: Interprofessional Team Member
EPA
Collaborate as interprofessional team member
Collect, Assess Plan, Implement, Follow-up: (Monitor & Eval)
1.1, 3.1, 3.3, 3.4, 3.6, 4.1, 4.2, 4.4

68. Domain 3: Pop. Health Promoter
EPA
Identify at-risk pts. for prevalent disease
Collect
2.3, 2.4,
3.2, 3.4, 3.5, 3.6
Minimize ADE & med errors
2.2, 2.3, 2.4
3.2, 3.3, 3.4, 3.5, 3.6
Maximize use of meds in population
2.1, 2.2, 2.3, 2.4
Ensure immunizations

69. Domain 4: Information Master
EPA
Educate pt & provider on medication use
Assess, Plan, Implement, Follow-up: M/E
1.1,
3.2, 3.3, 3.5, 3.6,
4.2, 4.4
Use evidence-based info to advance pt care
3.5, 3.6

70. Domain 5: Practice Manager
EPA
Oversee pharmacy operations at work
Assess, Plan, Implement, Follow-up: M/E
2.2
3.1, 3.6,
4.1, 4.2, 4.3, 4.4
Fulfill medication order
3.1, 3.6

71. Domain 6: Self-developer
EPA
Create written CPD plan
4.1, 4.4

72. Discussion Questions
Please share effective strategies for getting “Buy-in” from internal and external stakeholders
List specific tips for developing a comprehensive EPAs assessment plan?
Describe examples of “best practices” for designing, implementing, and assessing EPAs

73. Dr. Abby A. Kahaleh @ Akahaleh@Roosevelt.edu
Thank You!
Dr. Abby A.