Information Mastery: A Practical Approach to Evidence-Based Care Course Directors: Allen Shaughnessy, PharmD, MMedEd David Slawson, MD Tufts Health Care Institute Tufts University School of Medicine November 10-12, 2011 Boston, Massachusetts

Information Mastery: A Practical Approach to Evidence-Based Care Is It True? Evaluating Research about a Therapy

3 The new paradigm: probabilistic thinking Current paradigm: the biomedical model The body can be approached as an engineering problem External fetal monitoring Nelson KB, Dambrosia JM, Ting TY, Grether JK. Uncertain value of electronic fetal monitoring in predicting cerebral palsy. N Engl J Med 1996;334: Right heart catheterization Shah MR, et al. Impact of the pulmonary artery catheter in critically ill patients: meta-analysis of randomized clinical trials. JAMA Oct 5;294(13): The new paradigm: Probabilities What can we do for people that, on average, will help most of them most of the time?

4 What is Evidence-Based Medicine? “The judicious use of the best current evidence in making decisions about the care of the individual patient.” --EBM working group “An acknowledgment that there is a hierarchy of evidence and that conclusions related to evidence from controlled experiments are accorded greater credibility than conclusion grounded in other sorts of evidence.” -- Brian Hurwitz. BMJ 2004;329:

5 The Hierarchy of Evidence Results from controlled trials Results from case-control studies Results from case series Expert consensus or opinion Pathophysiologic reasoning Credibility

6 The Place of EBM in Medicine Goals of medicine: Relieve/prevent suffering; maintain/provide hope; prevent, treat, or cure disease The science of medicine: knowing the best way to prevent, treat, or cure disease (EBM can address this aspect) The art of medicine: Determining, using intuition, experience, and judgment, what patients need the most Combining the art and science  Clinical Jazz

7

8 Study Methods to Answer This Question Epidemiology: Patients taking a vitamin are less likely to have migraines Pharmacology: Drug x affects cerebral vasculature in rat brain isolates Case report: “It worked on one patient” Case-series: “It worked on a bunch of patients” Randomized controlled trial: 1/2 get drug, 1/2 placebo. No one knows who ‘til the end who took what

9 Validity Internal validity: How well was the study done? Do the results reflect the truth? External validity: can I apply these results to MY patients?

10 Was it a randomized controlled trial? Randomization is the best protection against being mislead

11 32 controlled trials of anticoagulation in acute MI Results by type of study: Chalmers TC, et al. N Engl J Med 1977;297: The value of randomization

12 Was allocation assignment “concealed”? Did investigators know to which group the potential subject would be assigned before enrolling them?

13 Importance of concealed allocation Trials with unconcealed allocation consistently overestimate benefit by ~40% Schulz KF, Chalmers I, Hayes RJ, et al. JAMA 1995;273: Schulz KF, Grimes DA. Lancet 2002;359: Pildal J, et al. Int J Epidemiol 2007;36: Moher D, et al. Lancet 1998;352:

14 Was allocation assignment “concealed”? Concealed allocation  blinding Blinding can occur without concealed allocation Surfactant in the NICU Allocation can be concealed in an unblinded study PT vs surgery for knee DJD Moseley JB, O'Malley K, Petersen NJ, et al. N Engl J Med 2002; 347:81-8.

15 Potential Subjects Conducting a Study Actual Subjects A B Randomization Blinding, etc Trial starts Concealed Allocation

16 Importance of concealed assignment Meta-analysis of trials evaluating screening mammography In studies in which allocation wasn’t concealed Higher SE status, education level in screened group Age disparity (average 6 mo older in the unscreened group) Richer, smarter, younger Trials with concealed allocation = screening harmful! No effect or increased mortality 20% more mastectomies Lancet Jan 8, 2000; Oct 20, 2001

Potential Subjects Conducting a Study Actual Subjects A B Randomization Blinding, etc Trial starts Concealed Allocation Mammography Study Sign-up Number Group Patient name 1 Mamm. 2No Mamm 3Mamm 4No Mamm 5No Mamm 6Mamm 7Mamm 8No Mamm 9Mamm 10No Mamm 11Mamm 12Mamm 13No mamm 14No mamm 15Mamm Sara Smith Jill Jones Wendy Walsh Linda Lucky

18 Technical Nitpicking? Could this really make a difference? Cumulative database: ~500,000 women Current policy is based on very small differences: Deaths in unscreened women902 Deaths in screened women837 Death difference (of 456,349) 65! Systematic bias is not “random error” for which meta- analysis can compensate

19 Mundus Vult Decipi “The world wishes to be deceived” People would rather be deceived than have the truth cause anxiety Caleb Carr, Killing Time

20 “YOU WANT ANSWERS??!!! “I WANT THE TRUTH!!” “YOU CAN’T HANDLE THE TRUTH!!” Jack Nicholson and Tom Cruise “A Few Good Men”

21 Nonfebrile Seizure Incidence

22 Were all the patients properly accounted for at its conclusion? Complete follow-up? “Intention to treat” analysis? Patients are analyzed in the groups to which they are assigned Attempts to reflect “real world” clinical situations in which not all patients follow treatment recommendations Watch when they compare only compliers with compliers and non- compliers Compliant subjects always do better overall

23 Was study “double-blinded”? Did the patients know to which group they were assigned? Did the treating physician know? Did investigators assessing outcomes know (“triple- blinding” – up to 7 levels!)? Judicial assessor blind + allocation concealment = surgery RCTs Schulz KF. Ann Int Med 2002;136:254-9.

24 Were intervention and control groups similar? See Table 1 of most studies Randomization is best way to avoid bias, though imbalances still can occur (especially if allocation was not concealed) Small differences sometimes are important