1. Daniel E. Ford, MD, MPH Vice Dean for Clinical Investigation Johns Hopkins University School of Medicine

2.  How much better are the outcomes?  Were the right outcomes measured?  What was the comparison group in the study?  Were the patients in the study representative of the broader population of patients for which the new therapy is targeted? Questions to Ask After the First Randomized Clinical Trial Demonstrates Efficacy of a New Therapy

3.  What kind of training would be required to provide the therapy described in the study?  How much of an investment is required to “try out” the new therapy?  How much does the new therapy cost?  Will payers cover the costs? What are the out-of-pocket costs?  Will providers adhere to the study protocol in the clinical trial? Questions to Ask After the First RCT Demonstrates Efficacy of a New Therapy

4.  Will patients adhere to the study protocol in the clinical trial?  After more widespread use, is the safety of the new therapy confirmed?  After more widespread use, is the effectiveness of the new therapy confirmed? Questions to Ask After the First RCT Demonstrates Efficacy of a New Therapy

5.  Warfarin has been used for years –does decrease cardiovascular events but needs monitoring, frequent dose adjustments, and has relatively high rate of bleeding complications  Dabigatran (Pradaxa) no need for monitoring and does not have a narrow window for therapeutic utility Dabigatran As New Anticoagulant

7. Figure 1 Country distribution of mean time in therapeutic range in the RE-LY trial Lars Wallentin, Salim Yusuf, Michael D Ezekowitz, Marco Alings, Marcus Flather, Maria Grazia Franzosi, Prem... Efficacy and safety of dabigatran compared with warfarin at different levels of international normalised ratio control for stroke prevention in atrial fibrillation: an analysis of the RE-LY trial The Lancet Volume 376, Issue 9745 2010 975 - 983 http://dx.doi.org/10.1016/S0140-6736(10)61194-4

8. Laboratory Research Clinical Research Population Research TRANSLATIONAL RESEARCH

9.  Efficacy: Does an intervention improve outcomes under ideal conditions?  Effectiveness: Does the intervention improve outcomes in routine or usual clinical care? Efficacy and Effectiveness

10. Table 1. Domains of the pragmatic–explanatory continuum indicator summary [20]. Chalkidou K et al. Clin Trials 2012;9:436-446 Copyright © by The Society for Clinical Trials

11.  Definition from the IOM Report on Priorities for CER  “The generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.” Comparative Effectiveness Research (CER)

12.  Clinical trials in everyday care settings ◦ Practice-based research networks ◦ Compare two active treatments and not placebo  Observational assessments ◦ Clinical research registries ◦ Administrative data (ex. CMS)  Evidence-synthesis Comparative Effectiveness Research

13.  Patients in ideal trial are different than usual care ◦ Less comorbidity ◦ More willing to accept side effects ◦ Not paying for the treatment ◦ More rigorous follow up to continue with treatment  Providers are different ◦ Better training ◦ Willing to follow protocol Why Does Efficacy Not Always Translate to Effectiveness?

14.  Clinical trials not that common  Using observational methods ◦ Large datasets with less depth ◦ Large sample needed to assess safety ◦ Most vexing issue is how to control for case-mix? Patient differences in who gets one intervention/treatment as compared to another  Propensity scores  Instrumental variables Evaluating Effectiveness

15.  Randomized 1663 patients with CHF to receive spironolactone or placebo  As part of study excluded all patients at high risk for elevated potassium  Study stopped early because found 30% reduction in risk of death  After study released, widely used in broader population of patients  Dramatic rise in rate of hyperkalemic- associated hospital admissions and deaths Randomized Aldactone Evaluation Study RALES

16.  Compare cardiac computed tomography (CCTA) to functional tests like exercise stress tests  150 centers and usual physicians interpreting results  Treatment based on tests is not protocol driven but based on clinicians judgement  Endpoints are death, MI, procedural complications, unstable angina, quality of life, resource use and cost effectiveness PROMISE trial – PROspective Multicenter Imaging Study for Evaluation of Chest Pain

17. Figure 1. Number of RCTs and percentage of pRCTs from 1990 to 2010.RCT: randomized controlled trial; pRCT: pragmatic randomized controlled trial. Chalkidou K et al. Clin Trials 2012;9:436-446 Copyright © by The Society for Clinical Trials

18.  Efficacy to Effectiveness  Comparative Effectiveness ◦ Deciding on best approach in usual care settings  Knowledge Implementation or Knowledge Transfer ◦ More quickly moving evidence-based approaches to all practice settings Moving to Ultimate Implementation

19.  Everett Rogers  Agricultural Cooperative Agents  Diffusion depends on: ◦ Relative advantage ◦ Compatibility ◦ Complexity ◦ Trialability ◦ Observability Diffusion Theory

20. Copyright ©2008 BMJ Publishing Group Ltd. Pronovost, P. J et al. BMJ 2008;337:a1714 Strategy for translating evidence into practice

21.  Costs ◦ Direct ◦ Indirect  Effectiveness ◦ Quality adjusted life years  Cost-effectiveness Costs(new) – Costs(old) / Effectiveness(new)-Effectiveness(old) Cost-Effectiveness

22.  Support clinicians who want to learn by reviewing the scientific literature and evaluate how they treat patients  Make clinical data available for analysis ◦ Best data are electronic and comprehensive  Provide access to biostatisticians  Encourage clinicians to participate in multicenter studies  Support RESEARCH on quality improvement Creating an Environment to Promote Comparative Effectiveness Research

23.  Online free courses ◦ http://ocw.jhsph.edu/http://ocw.jhsph.edu/ ◦ Epidemiology, Biostatistics, Health Economics  MPH with Certificate in Comparative Effectiveness Research  Institute for Clinical and Translational Research 2 week intensive course JHM Comparative Effectiveness Research Resources

24.  Goal of medical research does not stop when treatment/diagnostic test evaluated in academic centers  Need to always consider ultimate customers of research  A learning health system has to include comparative effectiveness research Conclusions