1. Andrew H Slattengren, DO University of Minnesota Medical School Department of Family Medicine and Community Health North Memorial Family Medicine Residency Program EFFoRT: Email Feedback For Resident Teaching

2. Disclosures I have nothing to disclose. I will not talk about any off-label uses of medications or devices. I have no significant financial interests to report. I have no conflicts of interest to report.

3. Objectives At the conclusion of this session, the participants should be able to: Cite the evidence behind the efficacy of the current education model for resident physicians in the ambulatory clinic Discuss the feedback model implemented at North Memorial Family Medicine Residency Program and results of that model

4. Problem Identification Education in ambulatory care clinics is characterized by variability, unpredictability, immediacy, and lack of continuity. Case discussions are short in duration, involve little teaching, and provide virtually no feedback (1).

5. General Needs Assessment Analysis of current performance Resident survey Faculty evaluation academic year 2012 Direct observation In our residency, faculty provide brief, verbal point-of-care feedback to residents regarding patients being presented Personal experience Direct positive feedback when supplementing precepting with email feedback method Literature review

6. Literature Review Some characteristics of excellent clinical teachers include: Role modeling desired behaviors Involving the learner in the teaching process Limiting the number of teaching points Giving feedback (2)

7. Literature Review Most important preceptor behaviors: Provide feedback Teaching style Create a supportive learning climate (3)

8. Literature Review Computer resources were more valued than books Reflecting a generation of learners who are comfortable accessing electronic information (3)

9. Literature Review A positive learning environment, with staff members as EBM role models, might therefore be one of the most important factors influencing the behavior of residents (4) Only clinically integrated teaching, however, has been reported to improve not only knowledge but also skills, attitudes, and behavior toward EBM (4)

10. Broad Program Goals Improve the educational experience provided to residents by precepting faculty in the family medicine continuity clinic Increase the use of EBM resources by faculty and residents in the continuity clinic Enhance the culture of patient-based learning in the continuity clinic

11. Objectives: Psychomotor Using informatics resources, precepting faculty will locate an answer to a clinically relevant question addressing a precepted patient within 4 minutes while sitting at the outpatient precepting table by the end of the academic year 2012/2013 Each precepting faculty will email one resident feedback during that precepting day including the answer to a clinically relevant question addressing a patient for every ½ day precepting in the academic year 2012/2012 and document completion of this email by ccing Slattengren

12. Objectives: Cognitive Participating faculty will list 5 online resources available to provide EBM answers to a clinically relevant question after participating in a faculty development session

13. Objectives: Affective Participating faculty will value this additional teaching interaction as measure by # of emails cced in the last ¼ of the academic year vs first ¼ of the academic year 2012/2013

14. Educational Strategies and Implementation FACULTY DEVELOPMENT SESSION July 2012 Demonstration – Case example Lecture – Evidence, resources available, standard work Simulation – Cases Small group discussion Post module test – list 5 online resources

15. Results Number of feedback emails (per 216 opportunities) Quarter 1285 Quarter 2 45 Quarter 3 85(reminder email sent to faculty) Quarter 4167

16. Results I am currently a: 20122013n (%) First year resident9 (40.9)7 (33.3) Second year resident7 (31.8)7 (33.3) Third year resident6 (27.3)7 (33.3)

17. Results The feedback provided by core faculty preceptors during my continuity clinic sessions at Broadway is an effective tool to assist my learning. 20122013 n (%) n (%) Strongly Disagree 0 0 Disagree 1 (4.5) 1 (5) Agree13 (59.1) 7 (35) Strongly Agree 8 (36.4)12 (60)

18. Results I review literature or other resources to answer patient-based clinical questions after my continuity clinic sessions. 20122013 n (%) n (%) Every time 0 0 A majority of the time13 (59.1)15 (71.4) Rarely 9 (49.1) 6 (28.6) Never 0 0

19. Results I feel comfortable using on-line resources to find answers to patient-based clinical questions. 20122013 n (%) n (%) Strongly Disagree 0 0 Disagree 1 (4.5) 1 (5.3) Agree14 (63.6)12 (63.2) Strongly Agree 7 (31.8) 6 (31.6)

20. Results “Good overall. One thing that's important is that preceptors are concise. As an intern, it's easy to be distracted by the amount of time a pt is waiting in the room...I remember almost nothing from lengthy precepting sessions.”

21. Results “Some faculty members consistently provide feedback during precepting while others never seem to.”

22. Results “Targeted articles sent from preceptors regarding a clinical situation seen that day are helpful reminders to read at home! Precepting at the end of 1st year is often time efficient- when the presentation of the patient is more concise, and time is spent on a clinical question.”

23. Results “It's nice to get one major teaching point in even if it's busy and not a lot of time to go into detail with the presentation- just depending on chief complaint.”

24. Results “I really like being sent articles after a day precepting (Dr. X is very good at this). I think it would be helpful if most preceptors could just mention one thing they think would be beneficial from the day to look into. I try to do this on my own, but once I leave clinic I'm often caught up in other things, or so ready to be done by the time I close charts that I just shut down. A little more motivation from faculty could be helpful.”

25. Results “I will admit that I do not do as much reading ex post facto as I should. I do read the follow-up emails that I get from Dr. X. Several months ago Dr.Y started to send these as well and said there was a push that every provider was aiming to do this. In my opinion this would be great. I do understand that we are learning to find resources four ourselves during this training period too and maybe a follow-up email could be some advice on a topic to search instead of sending a pdf file of a paper. This would facilitate learning by searching and possibly require synthesis of information rather than just putting a paper on a platter.”

26. Future areas of study Informatics didactic session Resident topic search prompted by faculty email Scholarly activity trends Ongoing faculty development

27. References 1. Irby D.M. Teaching and Learning in Ambulatory Care Settings: A Thematic Review of the Literature. Academic Medicine 1995, 70:10. 898-931. 2. Alguire P.C., DeWitt D.E, Pinsky L.E., Ferenchick G.S. Teaching in Your Office: A Guide to Instructing Medical Students and Residents, Second Edition. 40-41. 2008, American College of Physicians Press, Philadelphia. 3. Schultz K.W. et al. Medical Students’ and Residents’ preferred site characteristics and preceptor behaviours for learning in the ambulatory setting: a cross-sectional survey. BMC Medical Education 2004, 4:12 Published online 2004 August 6. doi: 10.1186/1472-6920-4-12. 4. van Dijk N., Hooft L., Wieringa-de Waard M. What Are the Barriers to Residents’ Practicing Evidence-Based Medicine? A Systematic Review. Academic Medicine 2010, 85:7. 1163-70. 5. A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8. Steinert Y. et al. Medical Teacher 2006, 28:6. 497-526. 6. Pluye P. et al. Evaluation of email alerts in practice: part 1 – review of the literature on clinical emailing channels. Journal of Evaluation in Clinical Practice 2010, 16. 1227-36. 7. Kogan J.R. et al. Have You Had Your Feedback Today? Academic Medicine 2000, 75:10. 1041. 8. Beidas S.O. Using E-mail as an Interactive Teaching Tool. Academic Medicine 2000, 75:6. 670.