Download presentation
Presentation is loading. Please wait.
Published byArleen Washington Modified over 9 years ago
1
EBM for Novice Maria Kwok, MD, MPH Assistant Clinical Professor Section of Pediatric Emergency Medicine Morgan Stanley Children’s Hospital of New York Presbyterian
2
Outline Why is EBM important? What is EBM Understand foreground vs. background questions Formulate PICO questions Find resources, do searches Validity of an article Appraise “Therapy” articles Appraise “Diagnostic” articles
3
Why is EBM Important? MEDLINE –400,000 new entries added each year To keep up-to-date –Need to read 6,000 articles each day
4
Why is EBM Important? Lag time from time of “knowing” to time of “implementation” –13 years for thrombolytic therapy –10 years for corticosteroids for acceleration of fetal lung maturity (Antman EM, JAMA, 1992)
5
Traditional Approaches Self-reported reading time per week. (University setting) –Medical students60 min. –Internsnone –Senior residents10 min. –Fellows45 min. –Attendings graduating Post 197560 min. Pre 197530 min.
6
There is simply no way we can keep up to date in medicine using traditional approaches!
7
Outline Why is EBM important? What is EBM? Understand foreground vs. background questions Formulate PICO questions Find resources, do searches Validity of an article Appraise “Therapy” articles Appraise “Diagnostic” articles
8
Quiz: Define EBM C) The conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients A) A means of camouflaging biostatistics in medical training B) An annoying and overused catch phrase
9
Research Evidence Clinical Expertise Patient Preference Decision Making
10
Evidence Based Medicine: A Process Patients Ask Questions SearchingAppraise
11
Validity Results Applicability
12
Outline Why is EBM important? What is EBM Understand foreground vs. background questions Formulate PICO questions Find resources, do searches Validity of an article Appraise “Therapy” articles Appraise “Diagnostic” articles
13
What’s the difference between foreground and background questions?
14
Asking the Precise Question Background questions Basic aspect of a disease Pathophysiology Etiology Basic treatment Who, what, when, how Foreground questions Specific knowledge Have 4 parts: Patient/problem Intervention Comparison intervention Clinical outcomes
15
Background or Foreground? What is asthma? Is prednisone helpful in asthma? What are the newest medication for asthma? Does atrovent used acutely make you feel better?
16
Background or Foreground? What is asthma? (B) What are the newest medication for asthma? (B) Does atrovent used acutely make you feel better? (F) Is prednisone helpful in asthma? (F or B) Foreground if compare to other drugs Background if interested in how it works
17
Outline Why is EBM important? What is EBM Understand foreground vs. background questions Formulate PICO questions Find resources, do searches Validity of an article Appraise “Therapy” articles Appraise “Diagnostic” articles
18
Structure of a Well-built Question Patient or population Be specific to capture the group you want Ex: Children w/ asthma Intervention Be specific Comparison group (if any) Compare to standard therapy or test Outcome Be precise What are the outcome of interest
19
Formulating the Clinical Question PICO
20
Can You Identify PICO? In children under 6 months, how does sleeping on back compared to sleeping on the stomach in terms of risk of SIDS? In children under 6 months (P), how does sleeping on back (I) compared to sleeping on the stomach (C) in terms of risk of SIDS (O)?
21
Can You Form a PICO Question? Clinical scenario: 5 yo with moderate persistent asthma now in severe acute asthma exacerbation. Intern gave 2 albuterol and orapred with minimal improvement. Intern asks why how good is atrovent?
22
Searchable PICO Question P: Population I: Intervention/diagnostic test/risk factor C: Comparison O: Outcome In children with acute asthma exacerbation (P), will the addition of atrovent (I) to albuterol (C) decrease the rate of hospitalization (O)?
23
Outline Why is EBM important? What is EBM Understand foreground vs. background questions Formulate PICO questions Find resources, do searches Validity of an article Appraise “Therapy” articles Appraise “Diagnostic” articles
24
Searching Superhero Best Bets
25
The Evidence Pyramid
26
If your question is about…Look for a … Intervention/Therapy Randomized controlled trial Diagnosis/Screening To assess the accuracy of the test To assess effect of test on health outcome Cohort study Randomized controlled trial PrognosisLongitudinal cohort Etiology/Risk factors Randomized controlled trial Cohort Case-control
27
Pre-appraised Resources Cochrane –http://www.cochrane.org/cochrane/revabstr/mainindex.htmhttp://www.cochrane.org/cochrane/revabstr/mainindex.htm –Very high quality reviews –Mostly questions of therapy National Guideline Clearinghouse –http://www.guideline.gov/http://www.guideline.gov/ –Guidelines of varying levels of quality –Do broad searches PEM database –http://researchinpem.homestead.com/homepage.htmlhttp://researchinpem.homestead.com/homepage.html –Not pre-appraised
28
Pre-appraised Resources Best Evidence Topics –http://www.bestbets.orghttp://www.bestbets.org –Developed in the ED of Manchester Royal Infirmary in UK –Usually EM topics –Free Clinical Evidence (CE Concise) –http://www.clinicalevidence.comhttp://www.clinicalevidence.com –From UK –Focus mostly on therapy –Free
29
Primary Search Engines Pubmed Ovid
30
Primary Search Engines Both PubMed and Ovid Contains MEDLINE and other bibliographic databases (Cochrane, EMBASE, etc) Sells access to the databases to libraries
31
PubMed MEDLINE vs. Ovid MEDLINE Same DATABASE Different SEARCH ENGINES Searching differently may get you slightly different results If you search both databases EXACTLY the same, your results will be the same
32
What is MEDLINE? MEDLINE is a DATABASE MEDLINE is produced by the National Library of Medicine MEDLINE is available free via Pubmed MEDLINE is sold to many vendors, like Ovid, who search the DATA in MEDLINE through different search engines
33
Database Producer Medline EMBASE CINAHL Vendors Ovid Pubmed Dialog LEXIS/NEXIS Personal PC Library PC
34
PubMed: Clinical Queries Pre-filtered searching Search on questions of –Therapy –Diagnosis –Etiology –Prognosis Or, search for Systematic Reviews Limit any of the categories to sensitivity (broad) or specificity (narrow)
38
Sensitivity vs. Specificity SPECIFICITY Narrower search More relevant items Possibility of losing some useful citations LESS information to weed through SENSITIVITY Broader search More irrelevant items Less chance of losing useful citations MORE information to weed through
43
OVID Most librarians favor this search engine Allow you to tailor your search
45
Design & Execute Search Strategy MeSH terms How articles are indexed Predefined categories from the National Library of Medicine Explode MeSH + all items underneath Focus MeSH only
46
Design & Execute Search Strategy Limiters Limit by languages, human, age, etc Hedges “Clinical trials” for therapy “Sensitivity” & “Specificity” for diagnostic tests “Cohort studies” for prognosis
47
More on Searching Tips “Term$” Search engines will look for occurrences of the word with any combination of letters following. (Ex: “Hospital$” will include hospitalization, hospitals, hospitalized) “Exp” for “explode” “Term.tw.” Allow search engine to look in the titles and abstract
48
More on Searching Tips “Term.af” Search all fields “..pg term” Perge/delete
49
For More Searching Tips Ask your librarians!
50
Outline Why is EBM important? What is EBM Understand foreground vs. background questions Formulate PICO questions Find resources, do searches Validity of an article Appraise “Therapy” articles Appraise “Diagnostic” articles
51
Appraise (Validity) http://www.cebm.utoronto.ca/teach/mate rials/dx.htmhttp://www.cebm.utoronto.ca/teach/mate rials/dx.htm http://www.cebm.utoronto.ca/teach/mate rials/therapy.htmhttp://www.cebm.utoronto.ca/teach/mate rials/therapy.htm
52
Validity of Therapy Articles Randomized? Blinded? Were the subjects similar at the start of the trial? Were all subjects treated the same except for the treatment of interest? Were all patients accounted for at the end of the trial?
53
Validity: 1. Were the Subjects Randomized? “Methods” section
54
Validity: 2. Were They Blinded? “Methods” section
55
Validity: 3. Were the Subjects Similar at the Start of the Trial? “Results” section
56
Validity: 4. Were All Subjects Treated the Same except for the Treatment of Interest? “Methods” section
57
Validity: 5. Were All Patients Accounted for at the End of the Trial? “Results” section
58
Validity of the Article Was the study randomized? Were the subjects blinded? Were the subjects similar at the start of the trial? Were all subjects treated the same except for the treatment of interest? Were all patients accounted for at the end of the trial?
59
Outline Why is EBM important? What is EBM Understand foreground vs. background questions Formulate PICO questions Find resources, do searches Validity of an article Appraise “Therapy” articles Appraise “Diagnostic” articles
60
Step 4: Appraise “Therapy” Study Therapy study –Number Needed to Treat (NNT) to prevent an adverse outcome Mathematically –NNT = 1/ARR (absolute risk reduction) ARR = Risk of adverse outcome in the control group (CER) – those in the experimental group (EER) –ARR* = CER** – EER*** *ARR: Absolute risk reduction **CER: control event rate ***EER: experimental event rate
61
More on NNT Variables must be dichotomous Cannot calculate NNT from continuous variables For NNT to be meaningful, the difference between the 2 groups must be significant
62
Let’s Calculate NNT CEREERARRNNT 50%20% 3%2.5% 30%25% CER: Control Event RateEER: Experimental Event Rate ARR: Absolute Risk Reduction NNT: Number Needed to Treat NNT=1/ARR 30%3 0.5% 200 5%20
63
Step 4: Appraise “Therapy” Articles ARR = CER – EER NNT = 1/ARR ARR = 52.6-37.5 = 15.1 NNT = 1/15.1 x 100 = 7
64
Step 5: Decision Making Atrovent –Low cost medication –Minimal to no side effects –Every 7 children with severe asthma exacerbation, 1 hospitalization will be prevented The benefits outweigh the risks. Therefore it would be worth treating the child with Atrovent in the emergency department.
65
Outline Why is EBM important? What is EBM Understand foreground vs. background questions Formulate PICO questions Find resources, do searches Validity of an article Appraise “Therapy” articles Appraise “Diagnostic” articles
66
Appraise (Validity) http://www.cebm.utoronto.ca/teach/mate rials/dx.htmhttp://www.cebm.utoronto.ca/teach/mate rials/dx.htm http://www.cebm.utoronto.ca/teach/mate rials/therapy.htmhttp://www.cebm.utoronto.ca/teach/mate rials/therapy.htm
67
Validity of Diagnostic Articles Was there an independent, blind comparison with a reference (“gold”) standard of diagnosis? Was the diagnostic test evaluated in an appropriate spectrum of patients (like those in whom it would be used in practice)?
68
Validity of Diagnostic Articles Was the reference standard applied regardless of the diagnostic test result? Was the test (or cluster of tests) validated in a second, independent group of patients?
69
Disease +Disease - Test +A True + B False + Test -C False - D True - A + B C + D A + CB + D
70
Sensitivity & Specificity Sensitivity The proportion of people with the target disorder (A+C) who have a positive test (A) A/(A+C) SnNout If a test has a high sensitivity, a negative result rules out the disease
71
Disease + Disease - Test + A True + B False + Test -C False - D True - A + B C + D A + C B + D
72
Sensitivity & Specificity Specificity The proportion of people without the disease (B+D) who have a negative test (D) D/(B+D) SpPin When a test has a high specificity, a positive result rules IN the diagnosis
73
Disease + Disease - Test +A True + B False + Test - C False -D True - A + B C + D A + C B + D
74
Predictive Values Positive Predictive Value Proportion of people with a positive test (A+B) who have the target disorder A/(A+B) Negative Predictive Value Proportion of people with a negative test (C+D) who are free of the disease (D) D/(C+D)
75
Disease +Disease - Test +A True + B False + Test -C False -D True - A + B C + D A + CB + D
76
What is Likelihood Ratio? A likelihood that a given test result would be expected in a patient w/ the target disorder compared w/ the likelihood that the same result would be expected in a patient w/o the target disorder
77
Likelihood Ratio Likelihood ratio (LR) of 1 means The test result identifies patients w/ the disorder at the same rate as identifying patients w/o the disorders LR >1 The test result identifies patients w/ the disorder at a faster rate; likelihood goes LR <1 The test result identifies patients w/ the disorder at a slower rate; likelihood goes
78
LR of 3 means: The result is 3 times more likely to occur in a patient w/ the target disease than in a patient without. In general, the LR for a test result is the percentage of “disease +” patients identified by that result divided by the percentage of “disease –” patients so identified.
79
Dis +Dis -Rate Dis + Rate Dis - LR Result Aab Result Bcd TotalXY What rate are Dis + patients identified by Result A? What rate are Dis - patients identified by Result A? What is the LR for Result A? a/Xb/Y (a/X) (b/Y) c/Xd/Y (c/X) (d/Y)
80
Disease +Disease - Test +A True + B False + Test -C False - D True - X = A + CY = B + D At what rate are Dis + patients identified by a “+ test” ? At what rate are Dis – patients identified by a “+ test”? What is the LR for a “positive” test result? A/X B/Y (A/X) ÷ (B/Y)
81
Positive Likelihood Ratio (LR +) LR + = sensitivity/(1 – specificity) = True pos rate/False pos rate
82
Disease +Disease - Test +A True + B False + Test -C False - D True - X = A + CY = B + D At what rate are Dis + patients identified by a “- test” ? At what rate are Dis – patients identified by a “- test”? What is the LR for a “negative” test result? C/X D/Y (C/X) ÷ (D/Y)
83
Negative Likelihood Ratio (LR-) LR - = (1 – sensitivity)/specificity = False neg rate/True neg rate
84
LR+LR- 1 – 2 V Poor 1 – 0.5 2 – 5Poor0.5 – 0.2 5 – 10Fair0.2 – 0.1 10 – 20Good0.1 – 0.05 > 20 V. Good < 0.05 Guide to Likelihood ratios
85
Why Do We Care About LR? Allow us to think about pre-test and post- test probabilities so that we can personalize the test to our patients
86
Test threshold Treatment threshold 0%100% No test needed Further testing needed Testing completed; treatment starts Test & Treatment Thresholds in the Diagnostic Process
87
Step 1: Pre-test Probability Prevalence The number of events (instances of a given disease or other condition) in a given population at a designated time. It is not a rate. It is used as pre-test probability It is the number of “disease +” patients to the “total”
88
Pre-test Probability 5 sources to determine this: Clinical experience Regional or national prevalence statistics Practice databases Original report on accuracy of the test Studies that calculate pre-test probability
89
Step 3: Calculate LR Pretest probability is 30% LR+ = ? Culture +Culture - Test +6518 Test -1072 Total7590 Rapid Strep LR + = TP rate/FP rate = (65/75) (18/90) = 4.35 Amir, 1994 LR - = FN rate/TN rate = (10/75) (72/90) = 0.16
90
Fagan Nomogram Pre-test probability on the left hand column LR in the middle Post-test probability on the right hand column
91
Translation You see a 4 yo child who comes in c/o sorethroat, no other complaints Your pre-test probability is that she has a 30% chance of having strep throat You wonder how good is rapid strep You look up an article (Amir, 1994) and find its sensitivity (86.5%) and specificity (80%)
92
Translation Your patient rapid strep is positive You calculated the LR + to be 4.35 You determined the post-test probability to be 66% Is that enough of a threshold for you to treat or would you rather do more test?
93
Test threshold Treatment threshold 0%100% No test needed Further testing needed Testing completed; treatment starts Test & Treatment Thresholds in the Diagnostic Process
94
A Different Article Showed… What if the LR+ is 12? What is the post-test probability?
96
“Diagnostic” Summary… Use the 2 x 2 table to determine likelihood ratio (LR) LR + = True pos rate/False pos rate LR - = False neg rate/True neg rate Use Fagan Nomogram
97
Outline Why is EBM important? What is EBM Understand foreground vs. background questions Formulate PICO questions Find resources, do searches Validity of an article Appraise “Therapy” articles Appraise “Diagnostic” articles
98
Knowledge is of two kinds; we know a subject ourselves; or we know where we can find information upon it. - Samuel Johnson
Similar presentations
© 2024 SlidePlayer.com Inc.
All rights reserved.