e-Learning Evidence-based Medicine Katrien Oude Rengerink Academic Medical Center Dept. of Obstetrics and Gynaecology The Netherlands Background: biomedical scientist with a masters in epidemiology at Radboud University Nijmegen. EBayesMet conference October 2011

Contents What is Evidence-based Medicine? Why e-learning modules EBM? e-learning basic principles EBM e-learning modules Teach-the-Teacher EBM

What is EBM? Dave Sackett BMJ 1996: “Good doctors use individual clinical expertise, the best available external evidence as well as patient preferences, and neither alone is enough.”

Reduction Number Needed to Read 120+ journals scanned 50,000 articles Is it valid? (<5%) Intervention: RCT Prognosis: inception cohort Etc Is it relevant? 6-12 GPs & specialists asked: Relevant? Newsworthy? < 0.5% selected www.evidence-basedmedicine.com Number Needed to Read is 200+

Why EBM? Exponential growth knowledge Integration research into clinical practice slow  evidence summaries important  skills for knowledge translation important

5 steps of EBM

Theory of the Stork vs Theory of Sexual Reproduction R2 0.49, p<0.05

Evidence supporting Theory of Stork Storks exist Unexplained features fetal development Sexual intercourse without delivery Correlation birth rate - stork population Storks not seen in hospitals: apparently medical interventions may replace storks EBM: hierarchy in evidence sources

Organisation of evidence

Parachute use to prevent death: systematic review of RCTs Parachutes used to prevent death and major injury due to gravitational challenge Free fall studies: no 100% mortality No RCT: basis use purely observational. Apparent efficacy potentially explained by a ‘healthy’ cohort effect. No evidence  do not use intervention  Some effects not need RCT

Why e-learning EBM? Clinical professionals value EBM important, but slow integration and application in practice Huge variation EBM content and education Europe Interactive, clinically integrated education best way learning and applying EBM Current available courses useful, but: - minimum integration clinical practice - low flexibility

An European e-learning course in Evidence Based Medicine Leonardo I An European e-learning course in Evidence Based Medicine Background: biomedical scientist with a masters in epidemiology at Radboud University Nijmegen. 12

Partners UK Germany - Agency for Quality in Medicine University of Birmingham/Birmingham Womens’ Hospital CASP International Germany - Agency for Quality in Medicine Poland - CASPolska Spain - CASPe Hungary - TUDOR Italy - Universita Cattolica del Sacro Cuore Switserland - Basel Institute for Clinical Epidemiology The Netherlands - AMC Independent steering commitee

Aim Familiarise participants with EBM basics to help incorporate evidence from systematic reviews on therapeutic interventions into daily clinical practice Harmonise EBM teaching across Europe Short, individual e-learning sessions On-the-job training

Course content 1. Defining clinical questions 2. Searching the evidence 3. Appraisal of the systematic review a. Validity of RCTs b. relative risk and 2x2 table 4. Applying evidence to patient 5. Implementation of evidence into practice 15

Instruments for evaluating education in EBM 4 EBP domains: Knowledge Skills Attitude Behavior enacting EBP steps in practice permorning evidence-based clinical manoevers affecting patient outcomes

Course evaluation 101 participants from Switserland, Germany, Hungary, Spain, UK Pre- and postcourse assessment Knowledge/skills: modelled Berlin Questionnaire Attitudes: questionnaire by Taylor L. et al.

Results Across all countries, knowledge scores improved from pre- to post-course for all five modules (p<0.001) (paired Wilcoxon signed ranks test) After completing course participants felt more confident that they can assess research evidence  E-learning at least equally effective as regular teaching

Leonardo II Evidence Based Medicine: Training the Trainers across the healthcare sector

Background During Leonardo I & Sicily EBM conference: need support teaching EBM in practice low level EBM teaching in clinical practice possibly due to lack of confidence to teach EBM in workplace Complementary to existing EBM programmes: exploit learning opportunities in clinical setting  Leonardo II

Partners England (lead): Italy – Universita Cattolica del Sacro Cuore University of Birmingham/Birmingham Women’s Hospital J&B Associates Italy – Universita Cattolica del Sacro Cuore Poland - CASPolska Hungary - TUDOR Switserland - Basel institute of clinical epidemiology Netherlands – AMC Germany – AzQ 22

Aim Train trainers to teach EBM through on-the-job training, by demonstration of applied EBM, to encourage clinically relevant teaching of EBM in post-graduate education Develop European standard in EBM education Target: clinical teachers who teach EBM in clinical practice to postgraduates

Course contents Based on teaching opportunities modules developed for 6 clinical settings: Learner-centered (e-learning) Problem-based Video clip incorporated Applicable in Europe 24

Teaching opportunities in clinical practice Learning Opportunities 5 EBM steps Formulating questions Searching for evidence Critical appraisal Integrating evidence with clinical scenario Bringing change to practice Ward round +++ (+) + ++ Journal club Clinical teaching and assessment Outpatient clinic Formal clinical meeting Audit Clinical audit provides the opportunity to apply all five of the EBM steps, in particular to bring change to clinical practice for the benefit of patient care

Example: Teaching and learning EBM during ward rounds Outline: Learning objectives Pre-requisites to undertake the module Learning opportunities EBM steps Example Video clip (3 min) Teaching and learning lesson Conclusion This slide presents the module outline. After we discuss the learning objectives for this module and the associated pre-requisite knowledge we will identify the learning opportunities that exist for EBM in ward rounds by going through the appropriate EBM steps on a clinical question where we have a specific video example from a real ward round. Following this we will explore the issues that relate to teaching and learning EBM during ward rounds and at the end we will provide some conclusions for this module. 26

Course evaluation 5 countries: about 10 clinical teachers who teach EBM to postgraduates per country piloted Pre- and postcourse assessment using ‘Basel questionnaire’  e-learning course on how to teach EBM in various settings is feasible and effective.

Barriers EBM teaching 120 clinical EBM teachers from 11 countries completed questionnaire regarding barriers for EBM teaching Barriers: lack of time in busy practice lack of requirements for EBM skills pyramid hierarchy in health care management structure.  Teaching EBM in clinical settings is most successful where EBM principles are culturally embedded and form part and parcel of everyday clinical decisions and medical practice.

Availability Free available in: English German French Polish Hungarian Accessible through: www.ebm-unity.org

References Sackett, BMJ 1996 EBM what is it and what isn’t it? Höfer T, Przyrembel H, Verleger S. New evidence for the Theory of the Stork. Paediatric and perinatal epidemiology. 2004(18):88-92. Smith GCS, Pell JP. Parachute use to prevent dath and major trauma related to gravitational challenge: systematic review of randomised controlled trials. BMJ 2003. Khan AT, Mehr MN, Gaynor AM, Bowcock M, Khan KS. Is general inpatient obstetrics and gynaecology evidence-based? A survey of practice with critical review of methodological issues. BMC Women’s Health 2006(6):5. Khan KS, Coomarasamy A. A hierarchy of effective teaching and learning to acquire competence in evidence-based medicine. BMC Med Educ 2006:15(6):59. Coppus SFPJ, Emperanza JI, hadley J et al. A clinically integrated curriculum in Evidence-based medicine for just-in-time learning through on-the-job training: The EU-EBM project. BMC Medical Education 2007(7):46. Shaneyfelt T, Baum KD. Instruments for evaluating education in evidence-based practice: a systematic review. JAMA 2006;296(9):1116-1127. Kulier R, Hadley J, Weinbrenner S et al. Harmonising Evidence-based medicine teaching: a study of the outomes of e-learning in five European countries. BMC Medical Education 2008(8):27. Taranganitam S, Barnfield G et al. Teaching trainers to incorporate evidence- based medicine (EBM) teaching in clinical practice: the EU-EBM project. Oude Rengerink K, Thangaratinam S. How can we teach EBM in clinical practice? An analysis of barriers to implementation of on-the-job EBM teaching and learning.

Thanks to Leonardo I Leonardo II Regina Kulier Shakila Thangaratinam Julie Hadley Susanne Weinbrenner Berrit Meyerrose Tamas Decsi Andrea Horvath Eva Nagy Jose Emparanza Sjors Coppus Theodoris Arvantis Amanda Burls Juan Cabello Marcin Kaczor Gianni Zanrei Karen Pierer Katarzyna Stawiarz Regina Kunz Ben Willem Mol Khalid Khan Steering committee Leonardo II Shakila Thangaratinam Gemma Barnfield Theodoros Arvanitis Susanne Weinbrenner Khalid Khan Berit Meyerrose Andrea Horvath Gianni Zanrei Regina Kunz Katja Suter Jacek Walczak Anna Kaleta Javier Zamora Harry Gee Ben Willem Mol Maciej Nogas Bernard Burnand Chantal Arditi

Katrien Oude Rengerink: K.OudeRengerink@amc.uva.nl www.ebm-unity.org