1. Using InfoPOEMS in Journal Club to Teach Evidence-Based Practice UMDNJ-New Jersey Medical School Robin Schroeder, MD Caryl Heaton, DO Judy Washington, MD Chantal Brazeau, MD January 26, 2007

2. Objectives By the end of the presentation, you will be able to: –List EBM resources to assist faculty learn and teach key concepts to student –Describe a process to begin Journal Club for EBP at your institution Including, Faculty Guide Web-based curriculum

3. In the beginning… We wanted to teach Evidence Based Practice so that it could be used in real world practice

4. Evolution of EBP in our Clerkship The Family Medicine Project 2001 In preceptor’s office, each student: –Selected one patient with both biomedical and psychosocial problem –Developed and delivered an evidenced-based patient education intervention –Completed a 7 page formal write-up about the experience (graded)

5. Evolution of EBP in our Clerkship Family Medicine Project 2005 Simple 2 page worksheet –PICO questions –Short paragraph descriptions of »Evidence »Intervention »Follow-up

6. Your Experience… How many of you have tried something like this? What were your results? Our students…

7. …Hated It!!! No matter what we did, they still hated it We adjusted the project each year Still, the most poorly rated portion of the clerkship

8. And Told Us So… Student Comments: –“basically irrelevant to anything” –“the project does not have a practical application in medicine” –“I absolutely abhorred the EBM project” –“just extra paperwork”

9. We Still Thought It Was Important… … and so did the LCME The site visitors: “grilled” us about the poor evaluations for the FM project Commended us for our work in EBP Encouraged us to keep trying despite the challenges

10. So We Tried Something Different Journal Club Format…with a twist Did not want to beat one article to death Wanted to use InfoPOEMS to teach EBP concepts –a scientifically sound AND efficient method to analyze scientific articles and their clinical application

11. What Did We Need to Do? Identify and Develop Curriculum What concepts did we want to teach? EBM POEM Randomization Bias Allocation concealment NNT/Absolute Risk How to REALLY read a paper

12. What Did We Need to Do? Identify resources and make them available to students –WebCT –Get our library involved Faculty Development –Adapted to faculty level

13. What Did We Need to Do? Facilitator’s Guide What needed to be included to make this doable for faculty as facilitators?

14. Curriculum Development We chose POEMS to answer clinical questions regarding our 20 common problems in family medicine Article type: –Treatment –Diagnostic test –Review (systematic and meta-analysis) –Guideline

15. Curriculum Development We arranged POEMs and original articles in sequential curricular order Basic concepts first: Randomization Bias Table 1, Figure 1 Allocation Concealment Intention to Treat

16. Curriculum Development We built on previous concepts with subsequent articles: Sensitivity/Specificity (SpPIN and SnNOUT) NNT & Absolute Risk Likelihood Ratios Test Accuracy & ROC curves Meta-Analysis & Forest Plots

17. EBM Content for Students One hour lecture prior to Journal Club –Also available on WebCT WebCT –EBM modules –Links to useful EBM web resources –Copies of important articles

18. Introduction to Journal Club and Review of Useful Statistics Caryl J. Heaton, D.O. NJMS-Dept Family Medicine

19. Key Principle: Definition Evidence Based Medicine  The conscientious, explicit, and judicial use of the best current evidence in making decisions about the care of individual patients  OR… more newly defined: The integration of best research evidence with clinical expertise and patient values best evidence clinical expertise patient's values EBM

20. Probability of disease given a positive test=Bayes theorem 1. (prevalence x sensitivity) ((prevalence x sensitivity) + ((1-prevalence) x (1-specificity))) Thomas Bayes 1702-1761 “how to make an inferrence from data to hypothesis”

21. There is an easier way – with Likelihood Ratios You can do a likelihood ratio in your head!

22. Number Needed to Treat NNT = 1/ARR The number of patients you must treat with the intervention for the time specified to prevent (or create) one additional outcome.

25. EBP Modules on WebCT 2004-2005 AcITAC Minigrant: Laura Byham-Gray, PhD, RD byhamgld@umdnj.edu byhamgld@umdnj.edu Judy Washington, MD, Linda Boyd, DO Roberta Bronson Fitzpatrick, MSLIS Diane Rigassio Radler, PhD, RD Riva Touger-Decker, PhD, RD Tricia Davidson, MS, RD

26. EBP Module Topics 1. Evidence Based Practice: The Basics 2. Developing a Clinical Question 3. Critical Appraisal of the Literature 4. Domains of Clinical Questions 5. Applying the Evidence into Clinical Practice

27. Educational Resources/Tutorials An Introduction to Information Mastery (Michigan State University) This is a Web-based course that introduces the basic concepts of Information Mastery, Evidence-Based Medicine (EBP), and critical appraisal of the medical literature. It was designed by Family Practice physician Mark Ebell, MD, MS. Suggested completion times are given for each section.An Introduction to Information Mastery Clinical Epidemiology for Effective Clinical Practice (PedsCCM and IntensiveCare.com, Children’s Hospital, Boston) This self-paced tutorial is set up like a workbook, with specific chapters devoted to various EBM topics. Estimated completion times are provided, which helps users to schedule their time.Clinical Epidemiology for Effective Clinical Practice The Clinician’s Lament: How Do I Keep Up With the Medical Literature? (NYAM EBM, Rosanne M. Leipzig) Downloadable PowerPoint presentations on various EBM topics.The Clinician’s Lament: How Do I Keep Up With the Medical Literature?

28. MORE Cochrane Library Training (University of York) This site contains a series of downloadable presentations and tutorials.Cochrane Library Training EMAC Clinical Solutions Web (Emory University Departments of Emergency Medicine and Internal Medicine) This EBM curriculum tackles topics such as expert searching and answering questions from the four basic domains.EMAC Clinical Solutions Web Evidence Based Medicine (University of Massachusetts School of Medicine and the Lamar Soutter Medical Library) This EBM site presents basic concepts in a modular format.Evidence Based Medicine Evidence-Based Medicine: Finding the Best Clinical Literature (University of Illinois – Peoria Library) This guide is designed to assist users in becoming experts in the use of biomedical information resources to answer clinical questions. It presents information on PICO, basic question types, resources from the literature, and helpful Web links.Evidence-Based Medicine: Finding the Best Clinical Literature

29. MORE Evidence-Based Medicine Toolkit (University of Alberta, Canada) A self-paced instructional program that contains useful worksheets and calculators.Evidence-Based Medicine Toolkit How to Find the Evidence: The Basics (ScHARR – School of Health and Related Research) This short course is designed for use by health professionals in the South and West region and aims to provide an overview of the information resources available to locate research evidence and the process of translating a clinical question into an effective search strategy. It takes approximately 90 minutes to complete.How to Find the Evidence: The Basics Introduction to Evidence-Based Medicine (Duke University) This tutorial is intended for any health care practitioner or student who needs a basic introduction to the principles of Evidence-Based Medicine. This tutorial is not designed to teach you how to search the literature.Introduction to Evidence-Based Medicine SUNY Downstate Medical Center Evidence-Based Medicine Tutorial (SUNY Downstate) Contains a self-paced tutorial that provides great basic information on EBM.SUNY Downstate Medical Center Evidence-Based Medicine Tutorial

30. MORE Syllabi for Practising EBM (CEBM – The Centre for Evidence-Based Medicine) This syllabus contains readings that are discipline-specific, e.g. evidence- based nursing, evidence-based occupational therapy, etc.Syllabi for Practising EBM The Systematic Literature Review: What it is and how it can help (University of Leeds, Liz Berry) An interactive primer of the basics of literature review.The Systematic Literature Review: What it is and how it can help Systematic Review of Effectiveness of Teaching Critical Appraisal (ICRF/NHS Centre for Statistics in Medicine) Read results of a study that seeks to assess if teaching critical appraisal skills has beneficial effects upon both health care workers and patient care.Systematic Review of Effectiveness of Teaching Critical Appraisal Teaching Evidence-Based Medicine (CEBM – The Centre for Evidence-Based Medicine) Contains links to useful teaching materials.Teaching Evidence-Based Medicine WISDOM Resource Database (The WISDOM Centre) This site presents a series of online tutorials, authored by Alan O’Rourke.WISDOM Resource Database

31. Important Articles

33. Facilitator’s Guide Completed worksheets for each article EBM statistics review, glossary “How to Read” articles –treatment –diagnostic tests –systematic reviews –meta-analysis –guidelines

34. On your CD…

36. Administrative Details Max 24 students per rotation –Two Journal Club groups (each group presents same articles) One facilitator per group –Each group meets twice for two hours each –Each student presents once –Therefore, 12 POEMS and articles are necessary

37. Administrative Details Each student, Is provided with all POEMS Is assigned one POEM and corresponding full article for presentation (otherwise they all want the shortest article) Finds article on-line Finds appropriate worksheet on WebCT

38. Faculty Development Extensive experience with EBP not necessary –Experience facilitating small group helpful EBP inexperienced “sit in” with more experienced facilitator

39. Faculty Development Perfect opportunity for research/PhD faculty to teach in clerkship (clinical plus research faculty members together) Materials in Facilitator Guide are sufficient for novice

40. Faculty Development On-line courses/modules Live courses –Tufts (Slawson & Shaughnessy) –Oxford Centre for Evidence-Based Medicine –others Books

41. How Has it Gone? 2005-06 Academic Year Faculty “experience” 2 faculty EBP knowledgeable 1 faculty in MPH classes 1 interested, self-taught surgeon 1 new to EBP faculty member

42. Facilitator Evaluation Sets Expectations Promotes meaningful constructive peer assessments Asks questions that stimulate group discussion 5 point Likert scale 4.2-4.5 4.2-4.7 4.3-4.6

43. Facilitator Evaluation Helps students recognize knowledge gaps Maintains enthusiasm and positive attitude Provides constructive feedback 5 point Likert scale 4.2-4.8 4.3-4.8 4.1-4.7

44. It’s Going Well! Overall evaluations between 4-5

45. They Like It!!! “I thought journal club ran very smoothly and I learned both about evaluating literature as well as the actual content of the articles” “I enjoyed journal club, especially in the POEMs format” “I would like to see this in more rotations”

46. They Like It!!! “Good learning tool for EBM” “It helped me realize the shortcomings of articles and that not all published articles are significant” “Excellent…I’m beginning to see how helpful evidence based medicine is.”

47. A Journal Club Session InfoPoem-all students read all POEMs Article-one student presents one article Worksheet- presenting student presents highlights Teaching Points-facilitator insures coverage via discussion

48. Evaluation of Students Pass/Fail To Pass: Be present for both sessions Be prepared for one’s own presentation Actively participate during all presentations –Engaged –Questioning –Listening

49. Evaluation of Students To Fail: Don’t do the above, but… nobody fails. A student who does not meet the requirements must present their article to a faculty member.

50. Let’s Try It… InfoPoem Popular diets equally effective for losing weight Clinical Question: Which of 4 popular diets (Atkins, Zone, Weight Watchers, and Ornish) is most effective for losing weight and reducing cardiac risk factors? Bottom Line: All 4 popular diets (Atkins, Zone, Weight Watchers, and Ornish) are equally effective for helping adults lose weight and reduce cardiac risk factors.

51. Article

52. Teaching Points-Relevance Do you care about this article? Will your patients care about it? Will it change your practice? If yes, how? If no, why not?

53. Figure 1

54. Teaching Points Randomized Controlled Trial (single-blinded) Randomization Concealed Allocation Figure 1 contents Bias

55. Table 1

56. Teaching Points Baseline characteristics of participants –Are they similar? P values What are they measuring? Intention to treat

57. Worksheet Take a look First one, fairly simple

58. Summary Important to teach EBP (in a manner that students like) –Journal Club model with a twist –POEMs Developed by FP’s for FP’s –Applicable to practice –Statistical vs. clinical significance

59. Summary All of our preceptors are now POEM literate Students see EBP JC as something that all clerkships should be doing Students recognize that all that is in print is not necessarily –Relevant –Good research Doable!

60. Questions?