1. Rural Palliative Care (PC) Education: Results of a Hybrid Course with Face-to- Face and Online Learning. Dr. Jose Pereira Alberta Cancer Foundation Professor of Palliative Medicine University of Calgary

2. Goals of the Project Instil residents with competencies required to care for terminally ill patients. Develop evaluation tools –knowledge, attitudes and skills. Explore residents’ responses to the inclusion in the curriculum of: –Spirituality, hope, suffering, self- awareness, use of narrative, humanities to engage affective domain of learning

4. Course Design Hybrid model: F2F & online OSCEs (x4) & F2F workshops 10 weeks Online 1½ days F2F workshops & OSCEs (x4): Course intro Technology intro. Communication Pain/Symptoms Interactive, schemes & case-based Communication Decision-making Ethics Pain/Symptoms Movies & arts

5. Course Design 10 weeks Online Assignments (E-mail) Case Discussions Thematic Discussions Ask the Expert 4 modules Each 2-weeks long Small group asynchronous discussions

6. Evaluation Framework Knowledge –Pre vs Post-course knowledge test –20-items, multiple choice questionnaire –Based on blueprint; face validity –Varying Bloom’s hierarchies Attitudes –Surveys –Self-perceived changes in clinical comfort levels 22 items, 5-point Likert-like scale (1=not at all comfortable, 5=very comfortable) –Inclusion of topics in learning –Focus Groups Skills –4 OSCEs Course participation Course itself –Surveys & Focus Groups

7. Results: Knowledge N=15 Internal Reliability: Cronbach’s  : 0.5 –(Need to increase # of items to 30 to increase reliability to 0.67) Significant improvement in knowledge –Repeated measures test: F=19.8, p=0.001 –Cohen’s effect size: 0.77 –Pre-course mean (SD): 12 (2.6) –Post-course mean (SD):16 (1.9)

8. Self-perceived clinical comfort levels: Pre vs Post course N=15 Significant improvement in comfort levels –Repeated measures test: F=75.3, p<0.001 –Cohen’s effect size: 0.92 –Pre-course mean (SD): 59.7 (10.9) –Post-course mean (SD): 82.8 (4.7)

9. Self-perceived comfort levels Pre versus Post Course –Little change in communication –Large change in pain & symptom management But At post course when asked “compared to when you first started…” –Large change in communication as well –Role of OSCEs for self-assessment

10. Focus Groups Results Ambivalence to including psychosocial care in case studies. –“..talk about one topic at a time; not mix; separate the psychosocial from the clinical” –“Would have liked to see more clinical stuff” [online] –“I don’t agree; the patient is a whole person, you cannot separate”

11. Focus Groups Results Ambivalence to spirituality in care  “Physicians should address spirituality when treating palliative patients…one cannot separate the physical and the spirit.”..but no- one has taught us how to do this  “For now, we want to learn more about fundamentals of medicine rather than spirituality”

12. How should we introduce spirituality? Perhaps in disguise

13. Possible Roles of OSCEs Education tool Needs Assessment Formative evaluation Summative evaluation

14. Developed from real cases 3 domains in each OSCE: physical issues, psychosocial issues & communication. (clinical decision-making & communication) Reflect major competencies  58 y/o university professor with breast cancer. Presents with cancer pain. Cancer pain management. Address fears of opioids, explore illness experience.  Young 32 y/o with advanced gastric cancer, nausea & vomiting from upper GI obstruction.Young children. Manage psychological distress, being in presence of suffering, managing nausea & vomiting.  60 Y/o man with severe shortness of breath from advanced ALS. Accompanied by wife. Explore fears, advanced planning & discuss code status, home care needs, manage dyspnea.  Office visit by home care nurse Interdisciplinary collaboration, manage delirium, inability to swallow & hypercalcemia in home setting 4 OSCEs in this Course

15. Steps in developing OSCEs 1.Identify competencies & blueprint 2.Develop OSCEs (as a team) 3.Review OSCEs with content experts & potential learners (sample from target group of learners.) 4.Prepare score sheets a.Checklist & Global Rating Scale. 5.Train actors & actresses 6.Prepare logistics for implementation. 7.Test OSCEs with actors/actresses 8.Do OSCEs (videotape) 9.Rehearse scoring with scorers 10.Preliminary reliability testing 11.Scoring 12.Modifying OSCEs.

16. Checklist vs Global Rating Scale? Opted for checklist & global rating Literature –Global rating scales scored by experts showed higher inter-station reliability, better construct & concurrent validity than did checklists. –The use of checklists prior to using a global ratings scale did not improve the reliability or validity of the global rating. Regehr G, et al. Acad Med 1998;73:993-997

17. Scale Design 1.Separate score sheet for each OSCE 2.Scale consists of two subscales: –Performance of Skill –Degree to which skill performed –Items rated on a 3-point scale 3.Criterion-based scoring [Doig et al; Thompson et al] –Omitted, performed but not competently, performed competently

18. Results Inter-rater Reliability based on 4 raters Cronbach’s Alpha 1.Performance of Skill:.77 -.88 2.Degree of Skill:.72 -.88 3.Overall Scale:.87 to.92 Further inter-rater and intra-rater reliability and generalizability being assessed.

19. What residents thought of OSCEs Very useful learning tools. Helped them identify their learning needs and provide them with practice. Would recommend it to other residents

20. Overall Course Evaluation Would recommend it to future residents Want practical approaches, not theoretical discussions Want more mentoring Some ambivalence about: –Online learning component –Thematic discussions Psychospiritual issues

21. Strengths & limitations Limitations –Small numbers limit generalisability

22. Conclusions There is a culture that does not value integrated care- need to address this in the undergraduate curriculum Evaluation methods require careful thought and expertise

24. Attitudes to Inclusion in Curriculum High levels of agreement (>4)*: –“An exploration of the nature of suffering should be included in the curriculum.” Moderate levels of agreement (3.7)*: –“Addressing spirituality as part of providing medical care should be included in undergraduate and postgraduate medical curricula.” –“Self-awareness is a competency that should be included in undergraduate and postgraduate medical curricula.” * 40% -67%: course had resulted in them agreeing more with the statement