Presentation is loading. Please wait.

Presentation is loading. Please wait.

542-10-#1 Statistics 542 Introduction to Clinical Trials Issues in Analysis of Randomized Clinical Trials.

Similar presentations

Presentation on theme: "542-10-#1 Statistics 542 Introduction to Clinical Trials Issues in Analysis of Randomized Clinical Trials."— Presentation transcript:

1 542-10-#1 Statistics 542 Introduction to Clinical Trials Issues in Analysis of Randomized Clinical Trials

2 542-10-#2 Issues in Analysis of Randomized Clinical Trials Reference: May, DeMets et al (1981) Circulation 64:669-673 Peto et al (1976) British Journal of Cancer

3 542-10-#3 Sources of Bias 1.Patient selection 2.Treatment assignment 3.Patient Evaluation 4.Data Analysis Methods to Minimize Bias 1.Randomized Controls 2.Double blind (masked) 3.Analyze what is randomized

4 542-10-#4 What Data Should Be Analyzed? Basic Intention-to-Treat Principle –Analyze what is randomized! –All subjects randomized, all events during follow-up Randomized control trial is the “gold” standard” Definitions Exclusions –Screened but not randomized –Affects generalizability but validity OK Withdrawals from Analysis –Randomized, but not included in data analysis –Possible to introduce bias!

5 542-10-#5 Patient Closeout ICH E9 Glossary –“Intention-to-treat principle - …It has the consequence that subjects allocated to a treatment group should be followed up, assessed, and analyzed as members of that group irrespective of their compliance with the planned course of treatment.”

6 542-10-#6 Intention To Treat (ITT) Principle Analyze all subjects randomized & all events Beware of “look alikes” –Modified ITT: Analyze subjects who get some intervention –Per Protocol: Analyze subjects who comply according to the protocol

7 542-10-#7 Patient Withdrawn in Analysis (1) Common Practice - 1980s –Over 3 years, 37/109 trials in New England Journal of Medicine published papers with some patient data not included Typical Reasons Given a.Patient ineligible (in retrospect) b.Noncompliance c.Competing events d.Missing data

8 542-10-#8 Patient Withdrawn in Analysis (2) A.Patient INELIGIBLE –After randomization, discover some patients did not in fact meet entry criteria –Concern ineligible patients may dilute treatment effect –Temptation to withdraw ineligibles –Withdrawl of ineligible patients, post hoc, may introduce bias

9 542-10-#9 Betablocker Heart Attack Trial (JAMA, 1982) 3837 post MI patients randomized 341 patients found by Central Review to be ineligible Results % Mortality PropranololPlacebo Eligible7.39.6 Ineligible6.711.3Best Total7.29.8  In the ineligible patients, treatment works best

10 542-10-#10 Anturane Reinfarction Trial (1980) NEJM Randomized, double blind, placebo controlled AnturanePlaceboTotal Randomized8138161629 Ineligible383371 Reasons for ineligible 1/3 -time since MI: 35 days 1/3 -enzymes not elevated 1/3 -other: age, enlarged heart, prolonged hospitalization,.… Number ineligible about the same in each treatment group BUT

11 542-10-#11 Anturane Reinfarction Trial (1980) 1629 patients randomized –1631 entered, but two patients randomized twice –Need to delete 03013, 17008 –Use first randomization! Declared post hoc 71 “ineligible” patients

12 542-10-#12 Anturane Reinfarction Trial (1980) PlaceboAnturaneTotal All8178121629 Ineligible333871 Eligible7847741558 < 7 day rule151530 Analyzable subjects7697591528 (Table 3) Analyzable Deaths - Within 7 days of being off drug

13 542-10-#13 1980 Anturane Mortality Results Anturane PlaceboP-Value Randomized74/813(9.1%)89/816(10.9%)0.20 “Eligible”64/775(8.3%)85/783(10.9%)0.07 “Ineligible” 10/38(26.3%)4/33(12.1%)0.12 P-Values for 0.0001 0.92 eligible vs ineligible Reference: Temple & Pledger (1980) NEJM, p. 1488

14 542-10-#14 1980 Anturane Mortality Results AnturanePlacebo Withdrawn104 “Early” discontinuation43 “Late” discontinuation61

15 542-10-#15 Total Mortality Anturane Reinfarction Trial (1980) I. All Pts - 1629 All DeathsRandomDeaths* NEJM accounts for only Anturane 81274106 Placebo 81789 43 Total1629 *163149 P = 0.20 (Table 3+6) II. 1558 Subjects - Exclude 71 Non-eligibles All DeathsRandomDeaths“71" Anturane 774 64(10) Placebo 784 85 (4) Total1558 149P = 0.07

16 542-10-#16 Total Mortality Anturane Reinfarction Trial (1980) III. 1528 Subjects - Exclude 71 Non-eligibles + 30<7 days All DeathsRandomDeaths Anturane 759 60 Placebo 769 80 Total1528 140 IV. 1528 Subjects Analyzable DeathsRandomDeaths Anturane75944 Placebo76962 Total1528 106 P = 0.076

17 542-10-#17 Total Mortality Anturane Reinfarction Trial (1980) Consider Patients Excluded I. < 7 day rule - 30 pts AliveDeadTotal Placebo10515 Anturane11415 Total21930

18 542-10-#18 Total Mortality Anturane Reinfarction Trial (1980) II. 71 "ineligibles" AliveDeadTotal Placebo29433 Anturane281038 Total571471

19 542-10-#19 Anturane Sudden Death (SD) I. All Patients (N = 1629) RandomizedNDA-SDsNEJM-SDs Placebo81641498 Anturane 81327303 Total1629 6879 P-value0.080.03 11 additional SD's were defined from submission of NDs to publications II. Exclude 71 Protocol Violators (N = 1558) RandomizedNDA-SDsNEJM-SDs Placebo 7834046 Anturane 7752628 Total1558 6664 P-value0.080.03 Difference of 8 SD's

20 542-10-#20 Anturane Sudden Death (SD) for Total Follow-up III. Exclude 71 Protocol Violators & 30 7 Day Rule Violators (N = 1528) RandomizedNDA-SDsNEJM-SDs Placebo7683737 Anturane 7602422 Total1528 6159 P-value0.090.04 * Information not necessarily given in NEJM articlebut used to prepare tables presented

21 542-10-#21 Anturane Analysis Percent Mortality Patient Group AnturanePlaceboP-value 1620 5.6 (45/806) + 7.5 (61/814)0.10 15675.2 (41/781)7.4 (58/786)0.07 15474.6 (35/768)7.4 (58/779)0.01 14754.4 (32/733)7.1 (53/742)0.02 Article '783.4 (25/733)5.9 (44/742)0.016 + Number of deaths/number at risk Table D - 1978 Article Comparison of the Mortality Experience for the 4 Patient Groups

22 542-10-#22 ART (NEJM, 1978) ART (NEJM, 1978) Comparisons of the Mortality Experience for the 73 Patients with "Objective" and "Subjective" Baseline Exclusions Groups Compared % Mortality Placebo vs. Anturane in the 73 ** 8.6 (3/35) * 26.3 (10/38) 73 vs. 1547 *** 17.8 (13/73) 6.0 (93/1547) (Both Treatment Groups) 73 vs. 1547 (Anturane Group)26.3 (10/38) 4.6 (35/768) 73 vs. 1547 (Placebo Group)8.6 (3/35)7.5 (58/779) * Number of deaths/number at risk ** 73 refers to the group of 73 patients with "objective" or "subjective" reasons at baseline for exclusion *** 1547 refers to the total group of randomized patients with the 73 patients with objective and subjective baseline exclusions removed

23 542-10-#23 ART (NEJM, 1978) ART (NEJM, 1978) P-Values Using Two Techniques for Survival Curve Comparisons of the Groups P-Values Groups Compared Mantel-Haenszel Gehan Method Method Placebo vs. Anturane in the 730.0450.052 73 vs. 1547 (Both Treatment Groups)0.00090.0003 73 vs. 1547 (Anturane Only)< 0.0001< 0.0001 73 vs. 1547 (Placebo Only)0.910.98

24 542-10-#24 Acceptable Policies For Ineligible Subjects 1.Delay randomization, confirm eligibility and allow no withdrawals (e.g. AMIS) (Chronic Studies) 2.Accept ineligibles, allow no withdrawals (e.g. BHAT, MILIS) (Acute Studies) 3.Allow withdrawals if: a. Procedures defined in advance b.Decision made early (before event) c.Decision independent and blinded d.Use baseline covariates only (two subgroups) e. Analysis done with and without

25 542-10-#25 B.WITHDRAWL FOR NON-COMPLIANCE References:Sackett & Gent (1979) NEJM, p. 1410 Coronary Drug Project (1980) NEJM, p. 1038 Two Types of Trials 1.Management -"Intent to Treat" Principle -Compare all subjects, regardless of compliance 2.Explanatory -Estimate optimum effect, understand mechanism -Analyze subjects who fully comply WITHDRAWALS FOR NON-COMPLIANCE MAY LEAD TO BIAS!

26 542-10-#26 Breast Cancer Adjuvant Therapy Probability of Disease Free Survival for Years Post Mastectomy (Method I) Redmond et al (1983) Cancer Treatment Report

27 542-10-#27 Breast Cancer Adjuvant Therapy Probability of Disease Free Survival for Years Post Mastectomy (Method II) Redmond et al (1983) Cancer Treatment Report

28 542-10-#28 Breast Cancer Adjuvant Trial Results using stratification by compliance analysis can be re-ordered according to definition Both previous graphs are for the placebo arm Lesson: Compliance is an outcome & analysis of one outcome, stratified by another, is highly vulnerable to bias

29 542-10-#29 Cancer Trial (5-FU & Radiation) Gastric Carcinoma Reference:Moertel et al. (Journal of Clinical Oncology, 1984) 62 patients randomized –No surgical adjuvant therapy vs. –5-FU and radiation 5 year survival results RandomizedPercent (%) Treatment23%P < 0.05 No Treatment4%

30 542-10-#30 Cancer Trial (5-FU & Radiation) Gastric Carcinoma According to treatment received 5 year survival Received % Survival Treatment20% Refused Treatment30%NS Control4%

31 542-10-#31 Example: Coronary Drug Project 5-Year Mortality Clofibrate Placebo N% Deaths N% Deaths Total (as reported)1103 20.02782 20.9 By Compliance1065 18.22695 19.4 < 80% 35724.6 88228.2 > 80% 70815.01813 15.1 Adjusting for 40 covariates had little impact Compliance is an outcome Compliers do better, regardless of treatment

32 542-10-#32 Example: Coronary Drug Project 2-Year Mortality Compliance Assessed Estrogen Placebo N% Deaths N% Deaths Total9036.223615.7 < 80%4886.14369.9 > 80%4156.319254.8 Comments Higher % of estrogens patients did not comply Beneficial to be randomized to estrogen & not take it (6.1% vs. 9.9%) Best to be randomized to placebo & comply (4.8%)

33 542-10-#33 Example: Wilcox et al (1980) Trial, BMJ 6-Week Mortality Propranolol Atenolol Placebo N% Deaths N% Deaths N% Deaths Total1327.61278.712911.6 Compliers883.4762.68911.2 Non-compliers4415.95117.64012.5 Comments Compliers did better than placebo Treatment non-compliers did worse than placebo Placebo non-compliers only slightly worse than compliers Analysis by compliers overestimates benefit

34 542-10-#34 Aspirin Myocardial Infarction Study (AMIS) % Mortality ComplianceAspirinPlacebo Good6.15.1 Poor21.922.0 Total10.99.7

35 542-10-#35 Summary of Compliance No consistent pattern ExampleNon-compliance Did Worse CDP Clofibrate, AMISBoth Treatment & Control CDP EstrogenControl Only Beta-blocker, WilcoxTwo Treatments, Not Control Compliance an outcome, not always independent of treatment Withdrawal of non-compliers can lead to bias Non-compliers dilute treatment Try hard not to randomize non-compliers

36 542-10-#36 II. Competing Events Subject may be censored from primary event by some other event (e.g. cancer vs. heart disease) Must assume independence If cause specific mortality used, should also look at total death If non-fatal event is primary, should also look at total death and non-fatal event Problem for some response measures

37 542-10-#37 III. Problem of Definitions ClassificationAnturanePlaceboP-value ART30/81248/8170.03 Another Committee 28/81239/8170.17 Cause specific definitions hard to apply Example: Anturane Reinfarction Trail (ART) (NEJM, 1980) Sudden Death

38 542-10-#38 Anturane Reinfarction Trial Sudden Death Category SourcePlaceboAnturaneP-value All patients & all NEJM48/81730/8120.03 sudden deaths AC39/81728/8120.17 "Eligible" patients & NEJM46/78528/7750.03 all sudden deaths AC37/78225/7730.12 Problem of cause specific definitions AC = Another review committee

39 542-10-#39 IV. "Wrong", Inconsistent, Outlying Data "Wrong" or "outlying" data may in fact be real Decisions must be made blind of group assignment All modifications or withdrawals must be documented

40 542-10-#40 V. Missing Outcome Data Design with zero –missingness may be associated with treatment for analysis, data are not missing at random even if same number missing, missing may be for different reason in each treatment group Implement with minimum possible Analyze exploring different approaches –if all, or most, agree, then more persuasive

41 542-10-#41 “Best” and “Worst” Case Analyses TreatmentControl Total Events170220 Lost to Follow-up 30 10 "Best" Case170230 "Worst" Case200220

42 542-10-#42 VI. Poor Quality Data

43 542-10-#43 Poor Quality Data (1) 1.Lost to Follow-up (enforced withdrawals)  NO DATA: PROBLEMS: –Not necessarily independent of treatment –Raises questions about study conduct

44 542-10-#44 Poor Quality Data (2) SOLUTIONS: 1.Keep to a minimum Easiest if vital status is the outcome Hardest if the response variables are time-related measures requiring a hospital or clinic visit 2.Censor at the time lost –Can be done in survival analysis –Assumes independence of treatment

45 542-10-#45 Poor Quality Data (3) SOLUTIONS: 3.Estimate missing data using previous data or averages 4.“Best” case and “worst” case analyses

46 542-10-#46 VII. Poor Clinic Performance in a Multicenter Study If randomization was stratified by clinic, then withdrawal of a clinic is theoretically valid Withdrawal must be done independent of the outcome at that clinic

47 542-10-#47 Mortality in Aspirin Myocardial Infarction Study (AMIS) Aspirin PlaceboP-value All 30 Centers246/2267219/22570.99 7 “Selected” Centers 39 66< 0.01 In “selected” centers, aspirin showed superiority

48 542-10-#48 Mortality in Beta-Blocker Heart Attack Trial (BHAT) PropranololPlaceboP-value All 32 Centers 138/1916188/1921< 0.01 Cox adjusted Z = 3.05 6 “Selected” Centers4326< 0.05 In “selected” centers, propranolol worse

49 542-10-#49 VIII. Special Counting Rules Events beyond a specified number of days after treatment stopped not counted "non-analyzable" Examples 1."7 Day Rule"Anturane (1978) NEJM 2."28 Day Rule"Timolol (1981) NEJM If used, must –Specify in advance –Be a long period to insure termination not related to outcome –Analyze results both ways

50 542-10-#50 IX. Fishing or Dichotomizing Outcomes Common practice to define a response (S,F) from a non-dichotomous variable By changing our definition, we can alter results Thus, definitions stated in advance Definitions should be based on external data

51 542-10-#51 Dichotomizing Outcomes Heart Rate Trt ATrt B SubjectPrePost  PrePost  17272072702 27473171683... 257373079790 Mean74.073.20.874.474.00.4 Example

52 542-10-#52 Three Possible Analyses (1) Change  Treatment ATreatment BP-Value 1.F = < 723250.49 S = > 720

53 542-10-#53 Three Possible Analyses (2) Change  Treatment ATreatment BP-Value 1.F = < 723250.49 S = > 720 2.F = < 519250.02 S = > 560

54 542-10-#54 Three Possible Analyses (3) Change  Treatment ATreatment BP-Value 1.F = < 723250.49 S = > 720 2.F = < 519250.02 S = > 560 3.F = < 317180.99 S = > 387

55 542-10-#55 X. Time Dependent Covariate Adjustment Classic covariate adjustment uses baseline prognostic factors only –Adjust for Imbalance –Gain Efficiency Adjustment by time dependent variates not recommended in clinical trials (despite Cox time dependent regression model) Habit from epidemiology studies

56 542-10-#56 Coronary Drug Project 5-Year Mortality Baseline Cholesterol % Deaths Cholesterol ChangeClofibratePlacebo < 250mg%*Fall16.021.2 < 250Rise25.518.7 > 250 mg%Fall18.120.2 > 250 **Rise15.521.3 Little change in placebo group Best to have a.Low cholesterol getting lower * b.High cholesterol getting higher ** Example

57 542-10-#57 Example: Cancer Trials A common practice to compare survival on patients with a tumor response Problem is that patient must first survive to be a responder length - bias sampling

58 542-10-#58 Cancer Trials (1) Advanced Breast Cancer: Surgery vs. Medicine Santen et al. (1981) NEJM (Letter to editor, Paul Meier, U of Chicago) A randomized clinical trial comparing surgical adrenalectomy vs. drug therapy in women with advanced breast cancer 17 pts withdrawn from surgery group 10 pts withdrawn from medical group

59 542-10-#59 Cancer Trials (2) Reasons –Medical group (10 pts) 2 stopped taking their drugs 5 drug toxicity –Surgical group (17 pts) 7 later refused surgery 8 rapid progression precluding surgery No follow-up data on these 27 pts presented

60 542-10-#60 XI. Subgroup Analyses

61 542-10-#61 False Positive Rates The greater the number of subgroups analyzed separately, the larger the probability of making false positive conclusions. No. of SubgroupsFalse Positive Rate 1.05 2.08 3.11 4.13 5.14 10.19

62 542-10-#62 Subgroup Analyses Focusing on a particular “significant” subgroup can be risky –Due to chance –Results not consistent Estimates not precise due to small sample size

63 542-10-#63 MERIT Total Mortality

64 542-10-#64 MERIT

65 542-10-#65 MERIT MERIT (AHJ, 2001)

66 542-10-#66 Praise I Praise I Ref: NEJM, 1996 Amlodipine vs. placebo NYHA class II-III Randomized double-blind Mortality/hospitalization outcomes Stratified by etiology (ischemic/non-ischemic) 1153 patients

67 542-10-#67 PRAISE I

68 542-10-#68 PRAISE I - Interaction Overall P = 0.07 Etiology by Trt Interaction P = 0.004 Ischemic P = NS Non-Ischemic P < 0.001

69 542-10-#69 PRAISE I - Ischemic

70 542-10-#70 PRAISE I – Non- Ischemic

71 542-10-#71 PRAISE II Repeated non-ischemic strata Amlodipine vs. placebo Randomized double-blind 1653 patients Mortality outcome RR  1.0

72 542-10-#72 Three Views Ignore subgroups and analyze only by treatment groups. Plan for subgroup analyses in advance. Do not “mine” data. Do subgroup analyses However view all results with caution.

73 542-10-#73 Analysis Issues Summary Important not to introduce bias into the analysis ITT principle is critical Important to have “complete” follow- up Off treatment is not off study

Download ppt "542-10-#1 Statistics 542 Introduction to Clinical Trials Issues in Analysis of Randomized Clinical Trials."

Similar presentations

Ads by Google