1. Statistics 542 Introduction to Clinical Trials Issues in Analysis of Randomized Clinical Trials

2. Issues in Analysis of Randomized Clinical Trials
Reference:
May, DeMets et al (1981)
Circulation 64:
Peto et al (1976)
British Journal of Cancer

3. Methods to Minimize Bias
Sources of Bias
1. Patient selection
2. Treatment assignment
3. Patient Evaluation
4. Data Analysis
Methods to Minimize Bias
1. Randomized Controls
2. Double blind (masked)
3. Analyze what is randomized

4. What Data Should Be Analyzed?
Basic Intention-to-Treat Principle
Analyze what is randomized!
All subjects randomized, all events during follow-up
Randomized control trial is the “gold” standard”
Definitions
Exclusions
Screened but not randomized
Affects generalizability but validity OK
Withdrawals from Analysis
Randomized, but not included in data analysis
Possible to introduce bias!

5. Patient Closeout ICH E9 Glossary
“Intention-to-treat principle - …It has the consequence that subjects allocated to a treatment group should be followed up, assessed, and analyzed as members of that group irrespective of their compliance with the planned course of treatment.”

6. Intention To Treat (ITT) Principle
Analyze all subjects randomized & all events
Beware of “look alikes”
Modified ITT: Analyze subjects who get some intervention
Per Protocol: Analyze subjects who comply according to the protocol

7. Patient Withdrawn in Analysis (1)
Common Practice s
Over 3 years, 37/109 trials in New England Journal of Medicine published papers with some patient data not included
Typical Reasons Given
a. Patient ineligible (in retrospect)
b. Noncompliance
c. Competing events
d. Missing data

8. Patient Withdrawn in Analysis (2)
A. Patient INELIGIBLE
After randomization, discover some patients did not in fact meet entry criteria
Concern ineligible patients may dilute treatment effect
Temptation to withdraw ineligibles
Withdrawl of ineligible patients, post hoc, may introduce bias

9. Betablocker Heart Attack Trial (JAMA, 1982)
3837 post MI patients randomized
341 patients found by Central Review to be ineligible
Results
% Mortality
Propranolol Placebo
Eligible
Ineligible Best
Total
 In the ineligible patients, treatment works best

10. Anturane Reinfarction Trial (1980) NEJM
Randomized, double blind, placebo controlled
Anturane Placebo Total
Randomized
Ineligible
Reasons for ineligible
1/3 - time since MI: < 25 days or > 35 days
1/3 - enzymes not elevated
1/3 - other: age, enlarged heart, prolonged hospitalization, .…
Number ineligible about the same in each treatment group
BUT

11. Anturane Reinfarction Trial (1980)
1629 patients randomized
1631 entered, but two patients randomized twice
Need to delete 03013, 17008
Use first randomization!
Declared post hoc 71 “ineligible” patients

12. Anturane Reinfarction Trial (1980)
Placebo Anturane Total
All
Ineligible
Eligible
< 7 day rule
Analyzable subjects
(Table 3)
Analyzable Deaths - Within 7 days of being off drug

13. 1980 Anturane Mortality Results
Anturane Placebo P-Value
Randomized 74/813 (9.1%) 89/816 (10.9%) 0.20
“Eligible” 64/775 (8.3%) 85/783 (10.9%) 0.07
“Ineligible” 10/38 (26.3%) 4/33 (12.1%) 0.12
P-Values for
eligible vs ineligible
Reference: Temple & Pledger (1980) NEJM, p. 1488

14. 1980 Anturane Mortality Results
Anturane Placebo
Withdrawn
“Early” discontinuation 4 3
“Late” discontinuation 6 1

15. Total Mortality Anturane Reinfarction Trial (1980)
I All Pts
All Deaths Random Deaths * NEJM accounts for only
Anturane
Placebo
Total *
P = (Table 3+6)
II Subjects - Exclude 71 Non-eligibles
All Deaths Random Deaths “71"
Anturane (10)
Placebo (4)
Total P = 0.07

16. Total Mortality Anturane Reinfarction Trial (1980)
III Subjects - Exclude 71 Non-eligibles + 30<7 days
All Deaths Random Deaths
Anturane
Placebo
Total
IV Subjects
Analyzable Deaths Random Deaths
Anturane
Placebo
Total
P = 0.076

17. Total Mortality Anturane Reinfarction Trial (1980)
Consider Patients Excluded
I. < 7 day rule - 30 pts
Alive Dead Total
Placebo
Anturane
Total

18. Total Mortality Anturane Reinfarction Trial (1980)
II "ineligibles"
Alive Dead Total
Placebo
Anturane
Total

19. Anturane Sudden Death (SD)
I. All Patients (N = 1629)
Randomized NDA-SDs NEJM-SDs
Placebo
Anturane
Total
P-value
11 additional SD's were defined from submission of NDs to publications
II Exclude 71 Protocol Violators (N = 1558)
Placebo
Anturane
Total
P-value
Difference of 8 SD's

20. Anturane Sudden Death (SD) for Total Follow-up
III. Exclude 71 Protocol Violators & 30 7 Day Rule Violators
(N = 1528)
Randomized NDA-SDs NEJM-SDs
Placebo
Anturane
Total
P-value
* Information not necessarily given in NEJM articlebut used to prepare tables presented

21. Comparison of the Mortality Experience for
Anturane Analysis
Table D Article
Comparison of the Mortality Experience for
the 4 Patient Groups
Percent Mortality
Patient Group Anturane Placebo P-value
(45/806) (61/814) 0.10
(41/781) 7.4 (58/786) 0.07
(35/768) 7.4 (58/779) 0.01
(32/733) 7.1 (53/742) 0.02
Article ' (25/733) 5.9 (44/742)
+ Number of deaths/number at risk

22. ART (NEJM, 1978) Comparisons of the Mortality Experience for the 73 Patients with "Objective" and "Subjective" Baseline Exclusions
Groups Compared % Mortality
Placebo vs. Anturane in the 73** 8.6 (3/35)* 26.3 (10/38)
73 vs. 1547*** (13/73) 6.0 (93/1547)
(Both Treatment Groups)
73 vs (Anturane Group) 26.3 (10/38) 4.6 (35/768)
73 vs (Placebo Group) 8.6 (3/35) 7.5 (58/779)
* Number of deaths/number at risk
** 73 refers to the group of 73 patients with "objective" or
"subjective" reasons at baseline for exclusion
*** 1547 refers to the total group of randomized patients with the 73 patients
with objective and subjective baseline exclusions removed

23. ART (NEJM, 1978) P-Values Using Two Techniques for Survival Curve Comparisons of the Groups
Groups Compared Mantel-Haenszel Gehan
Method Method
Placebo vs. Anturane in the
73 vs. 1547
(Both Treatment Groups)
73 vs (Anturane Only) < <
73 vs (Placebo Only)

24. Acceptable Policies For Ineligible Subjects
1. Delay randomization, confirm eligibility and allow no withdrawals (e.g. AMIS) (Chronic Studies)
2. Accept ineligibles, allow no withdrawals
(e.g. BHAT, MILIS) (Acute Studies)
3. Allow withdrawals if:
a. Procedures defined in advance
b. Decision made early (before event)
c. Decision independent and blinded
d. Use baseline covariates only (two subgroups)
e. Analysis done with and without

25. WITHDRAWALS FOR NON-COMPLIANCE
B. WITHDRAWL FOR NON-COMPLIANCE
References: Sackett & Gent (1979) NEJM, p. 1410
Coronary Drug Project (1980) NEJM, p. 1038
Two Types of Trials
1. Management
- "Intent to Treat" Principle
- Compare all subjects, regardless of compliance
2. Explanatory
- Estimate optimum effect, understand mechanism
- Analyze subjects who fully comply
WITHDRAWALS FOR NON-COMPLIANCE
MAY LEAD TO BIAS!

26. Breast Cancer Adjuvant Therapy Probability of Disease Free Survival for Years Post Mastectomy (Method I)
Redmond et al (1983) Cancer Treatment Report

27. Breast Cancer Adjuvant Therapy Probability of Disease Free Survival for Years Post Mastectomy (Method II)
Redmond et al (1983) Cancer Treatment Report

28. Breast Cancer Adjuvant Trial
Results using stratification by compliance analysis can be re-ordered according to definition
Both previous graphs are for the placebo arm
Lesson: Compliance is an outcome & analysis of one outcome, stratified by another, is highly vulnerable to bias

29. Cancer Trial (5-FU & Radiation) Gastric Carcinoma
Reference: Moertel et al. (Journal of Clinical Oncology, 1984)
62 patients randomized
No surgical adjuvant therapy
vs.
5-FU and radiation
5 year survival results
Randomized Percent (%)
Treatment 23% P < 0.05
No Treatment 4%

30. Cancer Trial (5-FU & Radiation) Gastric Carcinoma
According to treatment received 5 year survival
Received % Survival Treatment 20%
Refused Treatment 30% NS Control 4%

31. Example: Coronary Drug Project 5-Year Mortality
Clofibrate Placebo
N % Deaths N % Deaths
Total (as reported)
By Compliance
< 80%
> 80%
Adjusting for 40 covariates had little impact
Compliance is an outcome
Compliers do better, regardless of treatment

32. Example: Coronary Drug Project 2-Year Mortality
Compliance
Assessed Estrogen Placebo
N % Deaths N % Deaths
Total
< 80%
> 80%
Comments
Higher % of estrogens patients did not comply
Beneficial to be randomized to estrogen & not take it
(6.1% vs. 9.9%)
Best to be randomized to placebo & comply (4.8%)

33. Example: Wilcox et al (1980) Trial, BMJ 6-Week Mortality
Propranolol Atenolol Placebo
N % Deaths N % Deaths N % Deaths
Total
Compliers
Non-compliers
Comments
Compliers did better than placebo
Treatment non-compliers did worse than placebo
Placebo non-compliers only slightly worse than compliers
Analysis by compliers overestimates benefit

34. Aspirin Myocardial Infarction Study (AMIS)
% Mortality
Compliance Aspirin Placebo
Good
Poor
Total

35. Summary of Compliance No consistent pattern
Example Non-compliance Did Worse
CDP Clofibrate, AMIS Both Treatment & Control
CDP Estrogen Control Only
Beta-blocker, Wilcox Two Treatments, Not Control
Compliance an outcome, not always independent of treatment
Withdrawal of non-compliers can lead to bias
Non-compliers dilute treatment
Try hard not to randomize non-compliers

36. II. Competing Events
Subject may be censored from primary event by some other event (e.g. cancer vs. heart disease)
Must assume independence
If cause specific mortality used, should also look at total death
If non-fatal event is primary, should also look at total death and non-fatal event
Problem for some response measures

37. III. Problem of Definitions
Cause specific definitions hard to apply
Example: Anturane Reinfarction Trail (ART)
(NEJM, 1980)
Sudden Death
Classification Anturane Placebo P-value
ART 30/ /
Another Committee 28/812 39/

38. Anturane Reinfarction Trial Sudden Death
Category Source Placebo Anturane P-value
All patients & all NEJM 48/ /
sudden deaths AC 39/ /
"Eligible" patients & NEJM 46/ /
all sudden deaths AC 37/ /
Problem of cause specific definitions
AC = Another review committee

39. IV. "Wrong", Inconsistent, Outlying Data
"Wrong" or "outlying" data may in fact be real
Decisions must be made blind of group assignment
All modifications or withdrawals must be documented

40. V. Missing Outcome Data Design with zero
missingness may be associated with treatment
for analysis, data are not missing at random
even if same number missing, missing may be for different reason in each treatment group
Implement with minimum possible
Analyze exploring different approaches
if all, or most, agree, then more persuasive

41. “Best” and “Worst” Case Analyses
Treatment Control
Total Events
Lost to Follow-up
"Best" Case
"Worst" Case

42. VI. Poor Quality Data

43. Poor Quality Data (1) Lost to Follow-up
(enforced withdrawals)  NO DATA:
PROBLEMS:
Not necessarily independent of treatment
Raises questions about study conduct

44. Poor Quality Data (2) SOLUTIONS: Keep to a minimum
Easiest if vital status is the outcome
Hardest if the response variables are time-related measures requiring a hospital or clinic visit
Censor at the time lost
Can be done in survival analysis
Assumes independence of treatment

45. Poor Quality Data (3) SOLUTIONS:
3. Estimate missing data using previous data or averages
4. “Best” case and “worst” case analyses

46. VII. Poor Clinic Performance in a Multicenter Study
If randomization was stratified by clinic, then withdrawal of a clinic is theoretically valid
Withdrawal must be done independent of the outcome at that clinic

47. Mortality in Aspirin Myocardial Infarction Study (AMIS)
Aspirin Placebo P-value
All 30 Centers 246/ /
7 “Selected” Centers < 0.01
In “selected” centers, aspirin showed superiority

48. Mortality in Beta-Blocker Heart Attack Trial (BHAT)
Propranolol Placebo P-value
All 32 Centers / /1921 < 0.01
Cox adjusted Z = 3.05
6 “Selected” Centers < 0.05
In “selected” centers, propranolol worse

49. VIII. Special Counting Rules
Events beyond a specified number of days after treatment stopped not counted "non-analyzable"
Examples
1. "7 Day Rule" Anturane (1978) NEJM
2. "28 Day Rule" Timolol (1981) NEJM
If used, must
Specify in advance
Be a long period to insure termination not related to outcome
Analyze results both ways

50. IX. Fishing or Dichotomizing Outcomes
Common practice to define a response (S,F) from a non-dichotomous variable
By changing our definition, we can alter results
Thus, definitions stated in advance
Definitions should be based on external data

51. Dichotomizing Outcomes
Example
Heart Rate
Trt A Trt B
Subject Pre Post  Pre Post 
...
Mean

52. Three Possible Analyses (1)
Change  Treatment A Treatment B P-Value
1. F = <
S = >

53. Three Possible Analyses (2)
Change  Treatment A Treatment B P-Value
1. F = <
S = >
2. F = <
S = >

54. Three Possible Analyses (3)
Change  Treatment A Treatment B P-Value
1. F = <
S = >
2. F = <
S = >
3. F = <
S = >

55. X. Time Dependent Covariate Adjustment
Classic covariate adjustment uses baseline prognostic factors only
Adjust for Imbalance
Gain Efficiency
Adjustment by time dependent variates not recommended in clinical trials (despite Cox time dependent regression model)
Habit from epidemiology studies

56. Coronary Drug Project 5-Year Mortality
Example
Baseline Cholesterol % Deaths
Cholesterol Change Clofibrate Placebo
< 250mg%* Fall
< 250 Rise
> 250 mg% Fall
> 250 ** Rise
Little change in placebo group
Best to have
a. Low cholesterol getting lower *
b. High cholesterol getting higher **

57. Example: Cancer Trials
A common practice to compare survival on patients with a tumor response
Problem is that patient must first survive to be a responder
length - bias sampling

58. Cancer Trials (1) Advanced Breast Cancer: Surgery vs. Medicine
Santen et al. (1981) NEJM
(Letter to editor, Paul Meier, U of Chicago)
A randomized clinical trial comparing surgical adrenalectomy vs. drug therapy in women with advanced breast cancer
17 pts withdrawn from surgery group
10 pts withdrawn from medical group

59. Cancer Trials (2) Reasons Medical group (10 pts)
2 stopped taking their drugs
5 drug toxicity
Surgical group (17 pts)
7 later refused surgery
8 rapid progression precluding surgery
No follow-up data on these 27 pts presented

60. XI. Subgroup Analyses

61. False Positive Rates
The greater the number of subgroups analyzed separately, the larger the probability of making false positive conclusions.
No. of Subgroups False Positive Rate

62. Subgroup Analyses
Focusing on a particular “significant” subgroup can be risky
Due to chance
Results not consistent
Estimates not precise due to small sample size

63. MERIT Total Mortality

64. MERIT

65. MERIT (AHJ, 2001)

66. Praise I Ref: NEJM, 1996 Amlodipine vs. placebo NYHA class II-III
Randomized double-blind
Mortality/hospitalization outcomes
Stratified by etiology (ischemic/non-ischemic)
1153 patients

67. PRAISE I

68. PRAISE I - Interaction Overall P = 0.07 Etiology by Trt Interaction
Ischemic P = NS
Non-Ischemic P < 0.001

69. PRAISE I - Ischemic

70. PRAISE I – Non- Ischemic

71. PRAISE II Repeated non-ischemic strata Amlodipine vs. placebo
Randomized double-blind
1653 patients
Mortality outcome
RR  1.0

72. Three Views Ignore subgroups and analyze only by treatment groups.
Plan for subgroup analyses in advance. Do not “mine” data.
Do subgroup analyses
However view all results with caution.

73. Analysis Issues Summary
Important not to introduce bias into the analysis
ITT principle is critical
Important to have “complete” follow-up
Off treatment is not off study