1 THE ROLE OF COVARIATES IN CLINICAL TRIALS ANALYSES Ralph B. D’Agostino, Sr., PhD Boston University FDA ODAC March 13, 2006

2 OUTLINE Setting of Randomized Clinical Trials General issues for the use of covariates in primary analyses Clinical trial scenarios for covariate analysis (including secondary and third level analyses) –From good to not-so-good practices Closing Comments

3 SETTING OF RANDOMIZED CLINICAL TRIAL Two treatments to be compared Subjects randomized to treatments Desire to test differences of treatments on a primary end point (e.g., time to death) Baseline data (covariates) collected on subject –Age, Gender, Severity, Location of Cancer, Clinic Site

4 GENERAL ISSUES FOR USE OF COVARIATES IN PRIMARY ANALYSIS 1.Randomization is assumed adequate to balance treatment groups and covariates not used in primary analysis 2.Covariates used to reduce variability (to increase precision) 3.Randomization may not balance treatment groups. Imbalances identified and these covariates put into primary analyses.

5 GOOD PRACTICES The following scenarios have the feature of a careful analysis plan stated clearly in the protocol or statistical analysis plan (SAP) that was developed before data set is locked

6 CLINICAL TRIAL SCENARIOS INCLUDING SECONDARY &THIRD LEVEL 1.PRIMARY ANALYSIS: RANDOMIZATION IS ASSUMED ADEQUATE TO BALANCE TREATMENT GROUPS AND COVARIATES NOT USED IN PRIMARY ANALYSIS A statistical test is performed comparing directly the two treatments with a statistical test that does not include covariate (e,g., log rank test on time to death (survival) analysis). This is the primary analysis.

7 GOOD: T 1 better than T 2 Events Plots over Study Time TREATMENT 1 TREATMENT 2 - Time from randomization in years

8 1. NO COVARIATES USED IN PRIMARY ANALYSIS SECONDARY LEVEL ANALYSIS examines covariates and subgroups to see consistency and sensitivity of overall results Covariate analysis. In survival analysis extend log rank test to Cox regression that adds one or more covariates at a time (e.g., treatment and age to examine age effect in presence of treatment)

Overall Gender Males females Age < 65 >65 Years with Condition Hazard Ratio yes no Location X NO YES 1 better 2 better Special subgroup Treatment really works

10 1. NO COVARIATES USED IN PRIMARY ANALYSIS THIRD LEVEL ANALYSIS Build a multivariate model that includes all important covariates and treatment This allows us to demonstrate consistency across variables and build a multivariate model that includes effect of treatment. It also can identify important subsets.

11 CLINICAL TRIAL SCENARIOS INCLUDING SECONDARY &THIRD LEVEL 2. PRIMARY ANALYSIS: COVARIATES USED TO REDUCE VARIABILITY (TO INCREASE PRECISION) A statistical test is performed comparing directly the two treatments with a statistical test that adjusts for preselected covariates (e,g., Cox regression on time to death (survival) analysis that adjusts for age or severity). This is the primary analysis.

12 2. PRE-SELECTED COVARIATES USED TO REDUCE VARIABILITY Example: Weight reduction study use baseline weight Example: Age, disease severity and location of cancer may be a priori selected for primary analysis. These variables may also have been used in randomization for stratification

13 2. PRE-SELECTED COVARIATES USED TO REDUCE VARIABILITY SECONDARY AND THIRD LEVEL ANALYSES ARE THE SAME AS BEFORE EXCEPT A-PRIORI SELECTED COVARIATES NEED TO BE CONSIDERED

14 CLINICAL TRIAL SCENARIOS INCLUDING SECONDARY &THIRD LEVEL 3.PRIMARY ANALYSIS RANDOMIZATION MAY NOT BALANCE TREATMENT GROUPS. IMBALANCES IDENTIFIED AND THESE COVARIATES PUT INTO PRIMARY ANALYSES Potential covariates on which imbalances may exist are identified in protocol and tested for significance and then put into primary analysis (e.g., severity, gender, age) if significant Control of alpha error rates should be considered

15 3. COVARIATES FOR WHICH IMBALANCES IDENTIFIED FIRST SECONDARY AND THIRD LEVEL ANALYSES ARE THE SAME AS BEFORE EXCEPT IDENTIFIED IMBALANCED COVARIATES NEED TO BE CONSIDERED

16 BAD PRACTICE The following has the feature that the PRIMARY ANALYSIS FAILED and then exploratory analyses (stated as planned or not) undertaken to identify covariates and determine treatment effects

17 PROBLEMS 1.Once primary analysis fails the alpha error has been used. There is no way of retrieving it with a secondary or exploratory analysis 2.The variability often associated with this practice is often so large that it adds nothing beyond original failed primary analysis

18 OVERALL TEST IS NOT SIGNIFICANT TECHNICALLY YOU CANNOT GO BEYOND THIS WITH ANY STATISTICAL STATEMENTS USEFUL TO LOOK AT COVARIATES AND SUBSETS AS EXPLORATORY ANALYSIS (NOT EVEN APPROPRIATE TO CALL IT A SECONDARY ANALYSIS)

19 PROBLEM: T1 not better Events Plots over Study Time TREATMENT 1 TREATMENT 2 - Time from randomization in years

20 SEARCH FOR COVARIATES Age ( > = 65 years vs < 65 years) ECOG Performance (1 or 2 vs 0) Tumor size (> 18.7 cm 2 or not) Disease state

21 ECOG Status EFFECT Status = 0 1 or 2 - Time from randomization in years

22 PROCEDURE Can look at covariates one at a time related to outcome variable or in conjunction with treatment In the end a multivariate analysis is done with covariates and treatment in model

23 TYPICAL RESULTS PRIMARY ANALYSIS –HR = 0.98,.95 CI (0.78 TO 1.24) COVARIATE ANALYSIS AFTER FAILED PRIMARY ANALYSIS –HR = 0.86,.95 CI (0.67 TO 1.10) TWO HRs NOT SIGNIFICANTLY DIFFERENT TWO CONFIDENCE INTERVALS OVERLAP (NO DIFFERENCE)

24 ANOTHER TYPICAL RESULT Primary analysis – HR = 0.98 (.78 to 1.24) One set of covariates –HR = 0.68 (.50 to.94) Another set of covariates –HR = 0.89 (.63 to 1.05) –WHAT DO WE BELIEVE?

25 CONCLUSION: STATISTICAL PROPERTIES OF ANALYSES If Primary analysis is significance Then as secondary analyses we can examine covariates and control error rates (that is, can control chance of identifying falsely significant covariates). Looking for joint relation of treatment and covariates to end point If number of covariates is unspecified, then analysis is exploratory even here

26 Conclusion: Statistical properties of analyses (continue) If Primary analysis is not significant, then we cannot control error rate (falsely identifying significant covariates and “saving” or improving treatment differences). We have used “alpha” on overall test Often do not get clarity from later analyses

27 Closing Comments Careful analysis plans can be stated for dealing with covariates. Error rates (level of significance) can be controlled even for considering covariates if structure of statistical approach is clearly stated. We must not confuse statistical procedures with covariates stated in a pre-specified manner from covariate analysis that follow after failed primary analysis and have no way of controlling statistical error rates