1. Assessment of IPE Initiative: Structure, Process, Outcome Part 2 Moderated by: Abby A. Kahaleh, BPharm, MS, PhD, MPH January 21, 2016

2. Contact Information Abby A. Kahaleh, BPharm, MS, PhD, MPH Curriculum SIG Chair 847-330-4537 (Phone) Akahaleh@Roosevelt.Edu

3. Presenters 1. Christine K O'Neil, PharmD, BCPS, CPG, FCCP Professor, Director of Curricular Development and IPE Mylan School of Pharmacy Duquesne University 2. Elena Umland, PharmD Associate Dean for Academic Affairs Professor of Pharmacy Practice Jefferson College of Pharmacy Thomas Jefferson University 3. Michelle Z. Farland, PharmD, BCPS, CDE Clinical Associate Professor University of Florida College of Pharmacy 4. Jennifer Danielson, PharmD, MBA, CDE Director of Interprofessional and Experiential Education & Assistant Professor University of Washington School of Pharmacy

4. Objectives  Describe key structure of IPE programs  Review current IPE assessment tools  Explain critical steps in assessing IPE outcomes  Share "lessons learned” in developing, implementing, and assessing IPE

5. Presenter I Christine K O'Neil, PharmD, BCPS, CPG, FCCP Professor Duquesne University Mylan School of Pharmacy

6. Creating Interprofessional Education Experiences  Interprofessional education (IPE): “When students from two or more professions learn about, from and with each other to enable effective collaboration and improve health outcomes. “ World Health Organization. Learning Together to Work Together for Health. Report of a WHO study group on multiprofessional education for health personnel: the team approach. http://whqlibdoc.who.int/hq/2010/WHO_HRH_HPN_10.3_eng.pdf. Accessed August 25, 2014.

7.  IPE is well-received and is a conduit for enabling knowledge and skills necessary for collaborative work  IPE is less able to positively influence attitudes and perceptions towards others  Evidence of IPE effectiveness is limited  Evidence does suggest that an interprofessional approach improves quality and decreases cost of care. Hammick M, Freeth D, Koppel I et al. A best evidence systematic review of interprofessional education. Med Teach 2007;29:735-51. Reeves S, Zwarenstein M, Goldman J et al. Interprofessional education: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews 2008, Issue1. Art. No.:CD002213. DOI:10.1002/14651858.CD002213.pub2. Evidence for IPE

8. Interprofessional Education Collaborative (IPEC)  Expert panel consisting of representatives from: – Association of Colleges of Nursing, American Association of Colleges of Osteopathic Medicine, American Association of Colleges of Pharmacy, American Dental Education Association, Association of American Medical Colleges, and Association of Schools of Public Health.  Competencies are general and flexible enough to be used by any profession: – Domain 1 – Value and Ethics for Interprofessional Practice – Domain 2 – Roles and Responsibilities – Domain 3 – Interprofessional Communication – Domain 4 – Teams and Teamwork  IPE learning experiences should be linked to one or more of these competencies. Interprofessional Education Collaborative Expert Panel. Core Competencies for Interprofessional Collaborative Practice: Report of an Expert Panel. Washington, D.C: Interprofessional Education Collaborative; 2011.

9.  Planning and teaching by an interprofessional mix of faculty  Link to a minimum of one learning objective (IPEC Competencies) Content relating to interprofessional competence is included and preferably threaded, throughout the course  At least one assignment that necessitates interprofessional group work Experiential courses may be designated as interprofessional.  Student participants in the course/experience represent at least two health professions. – Match students with equivalent levels of education for IPE experiences – IPE experience should be reflective of real practice connections between disciplines – IPE learning experiences should be optimized to achieve IPE outcomes for each of the programs involved and do not necessarily need to involve every health profession  Includes an assessment of growth in interprofessional competence. Key Elements of IPE Courses & Experiences

10. Curricular Considerations  Ideally, IPE experiences should be present in didactic, experiential, and co-curricular formats.  Present in each year of the professional curriculum to facilitate development of pharmacist as collaborators.  Incorporation of early IPE exposure in pre- professional programs is also an opportunity.

11. Curricular Themes  Communication skills  Health care ethics  Treatment adherence  Public health  Emergency preparedness  Medication errors; patient safety topics  Care of the patient with chronic illnesses or those in special populations. Buring SM, Bhushan A, Brazeau G, Conway S, Hansen L, Westberg S. Keys to successful implementation of interprofessional education: learning location, faculty development, and curricular themes. Am J Pharm Educ. 2009;73(4): Article 60. Meyer SM, Garr DR, Evans C. Advancing Interprofessional Clinical Prevention and Population Health Education. Curriculum Development Guide for Health Professions Faculty. http://c.ymcdn.com/sites/www.aptrweb.org/resource/collection/245F0E9A-CA95-47CB-BCC6- 3CB02C0E2EB4/APTR-HPCTF_IPE_Crosswalk_2013.pdf. Accessed August 25, 2014. http://c.ymcdn.com/sites/www.aptrweb.org/resource/collection/245F0E9A-CA95-47CB-BCC6- 3CB02C0E2EB4/APTR-HPCTF_IPE_Crosswalk_2013.pdf

12.  IPE is possible with any model of pharmacy education: – Fully integrated academic health center – Partially co-located program with pharmacy and other professions under a common university ownership – Partially co-located with pharmacy and other professions under different university components – Pharmacy with other health professions but no medical school – Pharmacy with no other health education programs on campus  More readily implemented in programs that have co-existing health professions schools  Partnerships with programs outside the school and creative teaching strategies with technology may facilitate IPE experiences in distant partnerships. Implementing IPE: Models

13. Assessment of IPE is Essential  Reflection of what the student has gained  Other assessment tools: – Readiness for Interprofessional Learning Scale (RIPLS) Parsell G, Bligh J. Med Educ. 1999;33:95-100 – Interdisciplinary Education Perception Scale (IEPS) Luecht RM et al. J Allied Health. 1990;19:181-91 – Attitudes Toward Health Care Teams Scale Heinemann GD et al. Eval Health Prof. 1999;22:123-42  Programmatic Outcomes – 42-item questionnaire developed using the IPEC competencies Dow AW, DiazGranados D, Mazmanian PE, Retchin SM. An exploratory study of an assessment tool derived from the competencies of the interprofessional education collaborative. J Interprof Care. 2014 Jul;28(4):299-304. DOI:10.3109/13561820.2014.891573. Epub 2014 Mar 4.

14. Assessment Tools for IPE Elena Umland, PharmD Jefferson School of Pharmacy Michelle Farland, PharmD, BCPS CDE University of Florida College of Pharmacy Reference: https://nexusipe.org/advancing/measurement-instrumentshttps://nexusipe.org/advancing/measurement-instruments

15. Kirkpatrick Assessment Model Level 4b: Benefits to patients/clients Improvements in health or well being of patients/clients Level 4a: Change in organizational practice Wider changes in the organization and delivery of care Level 3: Behavioral change Identifies individuals’ transfer of IP learning to their practice and setting Level 2b: Acquisition of knowledge & skills Including knowledge and skills linked to IP collaboration Level 2a: Modification of perceptions & attitudes Changes in reciprocal attitudes or perceptions between participant groups, Changes in perception or attitude towards the value and/or use of team approaches to caring for a client Level 1: ReactionLearners’ views on the learning experience and its IP nature Academia Practice Culture Change Need to be here We are here Danielson J, Willgerodt M. University of Washington Health Sciences. 2015

16. Tools to Assess Reaction Tool NameBrief DescriptionIndividual/ Team Assessment Curricular Location Readiness for interprofessional learning scale (RIPLS) Evaluates the readiness of health professions students for IPE. IndividualEarly; prior to the start of any IPE programming. Potential to repeat it to see improvement in readiness as initial distribution may serve as a baseline. Interdisciplinary education perception scale (IEPS) Assess student perceptions of interprofessional experiences. IndividualEarly with inaugural IPE programming. Consider repeating to evaluate changes in perceptions. Note that 12-, 17- and 18- item scales may serve different purposes.

17. Tools to Assess Modifications of Perceptions & Attitudes Tool NameBrief DescriptionIndividual/ Team Assessment Curricular Location Student perceptions of physician- pharmacist interprofessional clinical education (SPICE) Assesses pharmacy and medical student attitudes toward interprofessional clinical education. IndividualAs early as possible where interprofessional clinical education occurs. First year students following a required IPE activity or course. Attitudes toward healthcare teams scale (ATHCT) Assesses team member perceptions of the quality of care delivered by healthcare team; team member attitudes towards physician authority in teams and their control over information about patients. Individual/teamAs used to determine effect of interprofessional interventions, its placement at beginning and end of program. 20-item scale (versus 14-item scale) has benefit of evaluating domain of roles/responsibilities.

18. Tools to Assess Behavior Change Tool NameBrief DescriptionIndividual/ Team Assessment Curricular Location Interprofessional collaborator assessment rubric (ICAR) Observational tool to assess learner achievement of interprofessional competency domains IndividualAPPEs. Faculty or preceptors observe students in interprofessional activities over time. Consider use at end of week 1, towards middle and at end of clinical rotation. Interprofessional collaborative competency attainment survey (ICCAS) Self-assessment of achievement of interprofessional care competencies IndividualAPPEs. At start and end of clinical rotation. Also consider pre- and post- other IPE activities as evaluates all 4 IPEC domains.

19. Tools to Assess Behavior Change Tool NameBrief DescriptionIndividual/Team Assessment Curricular Location IPEC competency survey instrument Self-assessment of the achievement of the competencies defined by the Interprofessional Education Collaborative expert panel IndividualConclusion of APPE. May be introduced early as formative assessment for pre- /post-assessment. Collaborative practice assessment tool (CPAT) Assesses the views of team members in a collaborative care team on elements of collaboration TeamFollowing a longitudinal experience (Pre-APPE or APPE). Performance assessment for communication and teamwork tool set (PACT) Observational tool to assess teams during a live simulated scenario TeamTeam OSCE (pre-APPE or APPE)

20. Tools to Assess Change in Organizational Practice Tool NameBrief Description Individual/Team Assessment Curricular Location Index for interdisciplinary collaboration (IIC) Assesses aspects and levels of interprofessional collaboration within an organization Individual & Team Designed for a longitudinal work environment may be helpful with students completing numerous APPEs at the same institution or residency programs Survey of organizational attributes of primary care (SOAPC) Assesses healthcare providers’ perceptions of resources available to make changes in the patient care process in primary care settings Team Designed for longitudinal teams, may be helpful with students completing numerous APPEs at the same institution or residency programs

21. IPE Assessment Tool Application  Identify the outcomes you need/want to measure.  Review tools to identify those that best measure the outcomes and are feasible to implement.  Multiple evaluation methods will be needed – Quantitative & qualitative – Self-assessment & observation of behavior

22. Presenter IV Jennifer Danielson, PharmD, MBA, CDE Director of Interprofessional Education and Experiential Education Assistant Professor University of Washington School of Pharmacy

23. Kahaleh A, Danielson J, Franson K, et al. AJPE 2015; 79 (1): Article 6.

24. Exposure Immersion Practice Values Roles Communication Teamwork Patient Care Outcomes Population Health Outcomes Shadowing Simulations Clinical Placements/Experiences Service Learning Project-Based Learning Exposure Immersion Practice Integration Student Growth Case Exercises Active Learning Danielson J, Willgerodt M. UW Health Sciences. 2015 UW SOP IPE Curriculum Framework

25. UW SOP Assessment Framework Exposure Immersion Integration Attitudes Knowledge Skills Behaviors Practice Change Graduation = Team-based practice ready Reaction Teamwork Communication Roles Values Danielson J. University of Washington School of Pharmacy. 2015

26. Q&A

27. THANK YOU!

28. Discussion Questions 1. What types of IPE programs have you implemented at your college/school of pharmacy? 2. What are some of the challenges/opportunities of your IPE program? 3. Which assessment tools have you used to evaluate students’ performance outcomes? 4. What are your future plans for CQI?