1. e-Learning Evidence-based Medicine
Katrien Oude Rengerink
Academic Medical Center Dept. of Obstetrics and Gynaecology
The Netherlands
Background: biomedical scientist with a masters in epidemiology at Radboud University Nijmegen.
EBayesMet conference October 2011

2. Contents What is Evidence-based Medicine? Why e-learning modules EBM?
e-learning basic principles EBM
e-learning modules Teach-the-Teacher EBM

3. What is EBM?
Dave Sackett BMJ 1996: “Good doctors use individual clinical expertise, the best available external evidence as well as patient preferences, and neither alone is enough.”

4. Reduction Number Needed to Read
120+ journals scanned
50,000 articles
Is it valid? (<5%)
Intervention: RCT
Prognosis: inception cohort
Etc
Is it relevant?
6-12 GPs & specialists asked: Relevant? Newsworthy?
< 0.5% selected
Number Needed to Read is 200+

5. Why EBM? Exponential growth knowledge
Integration research into clinical practice slow
 evidence summaries important
 skills for knowledge translation important

6. 5 steps of EBM

7. Theory of the Stork vs Theory of Sexual Reproduction
R2 0.49, p<0.05

8. Evidence supporting Theory of Stork
Storks exist
Unexplained features fetal development
Sexual intercourse without delivery
Correlation birth rate - stork population
Storks not seen in hospitals: apparently medical interventions may replace storks
EBM: hierarchy in evidence sources

9. Organisation of evidence

10. Parachute use to prevent death: systematic review of RCTs
Parachutes used to prevent death and major injury due to gravitational challenge
Free fall studies: no 100% mortality
No RCT: basis use purely observational. Apparent efficacy potentially explained by a ‘healthy’ cohort effect.
No evidence  do not use intervention
 Some effects not need RCT

11. Why e-learning EBM?
Clinical professionals value EBM important, but slow integration and application in practice
Huge variation EBM content and education Europe
Interactive, clinically integrated education best way learning and applying EBM
Current available courses useful, but: - minimum integration clinical practice - low flexibility

12. An European e-learning course in Evidence Based Medicine
Leonardo I
An European e-learning course in Evidence Based Medicine
Background: biomedical scientist with a masters in epidemiology at Radboud University Nijmegen. 12

13. Partners UK Germany - Agency for Quality in Medicine
University of Birmingham/Birmingham Womens’ Hospital
CASP International
Germany - Agency for Quality in Medicine
Poland - CASPolska
Spain - CASPe
Hungary - TUDOR
Italy - Universita Cattolica del Sacro Cuore
Switserland - Basel Institute for Clinical Epidemiology
The Netherlands - AMC
Independent steering commitee

14. Aim
Familiarise participants with EBM basics to help incorporate evidence from systematic reviews on therapeutic interventions into daily clinical practice
Harmonise EBM teaching across Europe
Short, individual e-learning sessions
On-the-job training

15. Course content 1. Defining clinical questions
2. Searching the evidence
3. Appraisal of the systematic review
a. Validity of RCTs
b. relative risk and 2x2 table
4. Applying evidence to patient
5. Implementation of evidence into practice 15

17. Instruments for evaluating education in EBM
4 EBP domains:
Knowledge
Skills
Attitude
Behavior
enacting EBP steps in practice
permorning evidence-based clinical manoevers
affecting patient outcomes

18. Course evaluation
101 participants from Switserland, Germany, Hungary, Spain, UK
Pre- and postcourse assessment
Knowledge/skills: modelled Berlin Questionnaire
Attitudes: questionnaire by Taylor L. et al.

19. Results
Across all countries, knowledge scores improved from pre- to post-course for all five modules (p<0.001) (paired Wilcoxon signed ranks test)
After completing course participants felt more confident that they can assess research evidence
 E-learning at least equally effective as regular teaching

20. Leonardo II
Evidence Based Medicine: Training the Trainers across the healthcare sector

21. Background
During Leonardo I & Sicily EBM conference: need support teaching EBM in practice
low level EBM teaching in clinical practice possibly due to lack of confidence to teach EBM in workplace
Complementary to existing EBM programmes: exploit learning opportunities in clinical setting
 Leonardo II

22. Partners England (lead): Italy – Universita Cattolica del Sacro Cuore
University of Birmingham/Birmingham Women’s Hospital
J&B Associates
Italy – Universita Cattolica del Sacro Cuore
Poland - CASPolska
Hungary - TUDOR
Switserland - Basel institute of clinical epidemiology
Netherlands – AMC
Germany – AzQ 22

23. Aim
Train trainers to teach EBM through on-the-job training, by demonstration of applied EBM, to encourage clinically relevant teaching of EBM in post-graduate education
Develop European standard in EBM education
Target: clinical teachers who teach EBM in clinical practice to postgraduates

24. Course contents
Based on teaching opportunities modules developed for 6 clinical settings:
Learner-centered (e-learning)
Problem-based
Video clip incorporated
Applicable in Europe 24

25. Teaching opportunities in clinical practice
Learning Opportunities
5 EBM steps
Formulating questions
Searching for evidence
Critical appraisal
Integrating evidence with clinical scenario
Bringing change to practice
Ward round
+++
(+) + ++
Journal club
Clinical teaching and assessment
Outpatient clinic
Formal clinical meeting
Audit
Clinical audit provides the opportunity to apply all five of the EBM steps, in particular to bring change to clinical practice for the benefit of patient care

26. Example: Teaching and learning EBM during ward rounds
Outline:
Learning objectives
Pre-requisites to undertake the module
Learning opportunities
EBM steps
Example
Video clip (3 min)
Teaching and learning lesson
Conclusion
This slide presents the module outline. After we discuss the learning objectives for this module and the associated pre-requisite knowledge we will identify the learning opportunities that exist for EBM in ward rounds by going through the appropriate EBM steps on a clinical question where we have a specific video example from a real ward round. Following this we will explore the issues that relate to teaching and learning EBM during ward rounds and at the end we will provide some conclusions for this module. 26

28. Course evaluation
5 countries: about 10 clinical teachers who teach EBM to postgraduates per country piloted
Pre- and postcourse assessment using ‘Basel questionnaire’
 e-learning course on how to teach EBM in various settings is feasible and effective.

29. Barriers EBM teaching
120 clinical EBM teachers from 11 countries completed questionnaire regarding barriers for EBM teaching
Barriers:
lack of time in busy practice
lack of requirements for EBM skills
pyramid hierarchy in health care management structure.
 Teaching EBM in clinical settings is most successful where EBM principles are culturally embedded and form part and parcel of everyday clinical decisions and medical practice.

30. Availability Free available in: English German French Polish Hungarian
Accessible through:

31. References Sackett, BMJ 1996 EBM what is it and what isn’t it?
Höfer T, Przyrembel H, Verleger S. New evidence for the Theory of the Stork. Paediatric and perinatal epidemiology. 2004(18):88-92.
Smith GCS, Pell JP. Parachute use to prevent dath and major trauma related to gravitational challenge: systematic review of randomised controlled trials. BMJ 2003.
Khan AT, Mehr MN, Gaynor AM, Bowcock M, Khan KS. Is general inpatient obstetrics and gynaecology evidence-based? A survey of practice with critical review of methodological issues. BMC Women’s Health 2006(6):5.
Khan KS, Coomarasamy A. A hierarchy of effective teaching and learning to acquire competence in evidence-based medicine. BMC Med Educ 2006:15(6):59.
Coppus SFPJ, Emperanza JI, hadley J et al. A clinically integrated curriculum in Evidence-based medicine for just-in-time learning through on-the-job training: The EU-EBM project. BMC Medical Education 2007(7):46.
Shaneyfelt T, Baum KD. Instruments for evaluating education in evidence-based practice: a systematic review. JAMA 2006;296(9):
Kulier R, Hadley J, Weinbrenner S et al. Harmonising Evidence-based medicine teaching: a study of the outomes of e-learning in five European countries. BMC Medical Education 2008(8):27.
Taranganitam S, Barnfield G et al. Teaching trainers to incorporate evidence- based medicine (EBM) teaching in clinical practice: the EU-EBM project.
Oude Rengerink K, Thangaratinam S. How can we teach EBM in clinical practice? An analysis of barriers to implementation of on-the-job EBM teaching and learning.

32. Thanks to Leonardo I Leonardo II Regina Kulier Shakila Thangaratinam
Julie Hadley
Susanne Weinbrenner
Berrit Meyerrose
Tamas Decsi
Andrea Horvath
Eva Nagy
Jose Emparanza
Sjors Coppus
Theodoris Arvantis
Amanda Burls
Juan Cabello
Marcin Kaczor
Gianni Zanrei
Karen Pierer
Katarzyna Stawiarz
Regina Kunz
Ben Willem Mol
Khalid Khan
Steering committee
Leonardo II
Shakila Thangaratinam
Gemma Barnfield
Theodoros Arvanitis
Susanne Weinbrenner
Khalid Khan
Berit Meyerrose
Andrea Horvath
Gianni Zanrei
Regina Kunz
Katja Suter
Jacek Walczak
Anna Kaleta
Javier Zamora
Harry Gee
Ben Willem Mol
Maciej Nogas
Bernard Burnand
Chantal Arditi

33. Katrien Oude Rengerink: K.OudeRengerink@amc.uva.nl