Overall and subgroup analysis If the OVERALL results show highly significant evidence of a worthwhile effect of treatment, but a few subgroups of the overview unexpectedly indicate no benefit (which could well happen by chance), then the appropriate question is whether there is good evidence that this life-saving treatment should be denied to these patients. REVERSAL of the usual demand that there should be proof of worthwhile benefit. Courtesy of Dr. K. Wheatley

Meta-analysis vs. randomized controlled trials: internal validity vs. generalizibility Have complimentary roles –RCT, large adequately powered If our desire is to assess the efficacy of treatment (i.e. understand a measure of benefit of the treatment under ideal conditions of a clinical trial using narrow defined eligibility criteria) –Meta-analysis (of totality of evidence) If our goal is to obtain reliable estimate about the treatment effectiveness (i.e. understand the extent to which a given treatment can produce a beneficial effect under variety of circumstances and eligibility criteria)

Meta-analysis vs. randomized controlled trials Small CTs To study mechanisms Meta-analyses of small RCTs To generate hypotheses for more reliable RCTs Large RCts Meta-analyses of large RCTs To obtain reliable overall answers under specific conditions of a trial To obtain a typical and unbiased and generalizible estimate of treatment effect and to explore interactions among subgroups

Literature-based vs. individual patient data meta-analysis? IPD MA gold-standard LMA may be misleading –Data extraction, patient exclusion, length of follow-up, method analysis may be less accurate in LMA Lancet 1993;341:418-22; Stat Med 1998;14:

IV Ethical obligations to account of what’s already known To avoid unnecessary trials if reliable knowledge already exists Conversely, to determine if there is true uncertainty about relative values of competing treatment alternatives –A new trial should be conducted if there is a substantial uncertainty which of the trial treatments would benefit the patient better Requirement that equipoise (uncertainty principle) is met

Ethical obligation of building systematically on what is already known Clinical trials should be preceded by a systematic review and should be reported with a discussion of assessing the trial’s results in the context what is already known –Ethical requirement for updating systematic reviews UK, Denmark, Holland now mandates search or conduct of SR before a new clinical trial is done JAMA 1998;280: ;Lancet 2001:358:1648

V Knowledge resources

“It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials.” Archie Cochrane

Cochrane Database of Systematic Reviews - The Cochrane Collaboration - an international network of individuals and institutions committed to preparing, maintaining, and promoting the accessibility of systematic reviews of the effects of health care interventions. Cochrane Systematic Reviews (2,796) (January 2003) Database of Abstracts of Reviews of Effectiveness (3,875) Registry of Randomized Controlled Trials (353,809)

How many systematic reviews are needed to “cover” whole medicine? 10,000 systematic reviews to provide broad coverage of most health care topics Clarke M, personal communication

Cochrane Centres South African Australasian Chinese Brazilian Nordic German San Antonio ItalianIberoamerican French Dutch UK Canadian New England San Francisco

Cochrane Systematic reviews Cochrane reviews have been shown to be methodologically superior to non- Cochrane systematic reviews BMJ 2000;320:537-40, JAMA 1998;280:278-80

Cochrane Cancer Network with Update Software Ltd The Cancer Library Courtesy of Dr. Chris Williams

Meta-analyses in radiation oncology 100 meta-analyses in the Cochrane Database of Systematic Reviews –22 Cochrane Reviews –78 DARE reviews MEDLINE (Clinical Queries) search –616 systematic reviews

Meta-analyses in radiation oncology: an example of reliable review with long-term (20 years) follow-up Favourable and unfavourable effects on long-term survival of radiotherapy for early breast cancer: an overview of the randomised trials Early Breast Cancer Trialists' Collaborative Group* Lancet 2000; 355: 1757­70 (20 May 2000 )

Proportional effects on all-cause mortality in 40 trials of radiotherapy

Lancet 2000; 355: 1757­70 (20 May 2000 )Lancet 2000; 355: 1757­70 (20 May 2000 )

Absolute effects of radiotherapy on cause-specific survival

Absolute benefits and hazards

Part VI Evidence and decision-making

Clinical Decision Making Evidence from research Patient circumstances Preferences, values and rights Courtesy of Dr. G. Lyman

Reporting data on benefits and harms If evidence on benefits and harms are not reported or is of poor quality, one has to wonder how physicians make decisions and recommendations for their patients Eddy D. JAMA 1990;264:

Reporting data on benefits and harms: RCTs in myeloma Survival outcomes 111/136 (82%) Survival beyond 5 years 15/111 (14%) Treatment-related mortality 33/136 (24%) Non-fatal adverse events 91/136 (67%) Annals Oncol 2001;12:

Reporting harms in RTOG randomized trials N= 51N = 54 N = 44

HOW TO INTEGRATE BENEFITS AND RISKS OF AVAILABLE THERAPEUTIC OPTIONS Should we always use the option with the best benefit/risk ratio? Efficacy=80% Toxicity=10%E/R=8 Efficacy=20% Toxicity=1%E/R=20

Decision-making at the bedside Minimal conditions for treatment benefit at which therapy is worth considering is met when –Absolute benefits>absolute harms (adjusted for the probability of bad event, e.g. relapse) Never administer treatment or order diagnostic test if treatment harm is greater than its efficacy

Integrating benefits and harms of radiation therapy of breast cancer Threshold for administering radiation therapy (RT): probability of breast cancer recurrence (without RT)> Deaths due to (RT) (%) 4.3% (=4.8%) = 89.6% (actual relapse=30.1%) Deaths due to breast cancer without RT- deaths due to breast cancer on RT