Presentation on theme: "(C) Stephen Senn 20041 Change from baseline or analysis of covariance?: Lord's Paradox and other matters. Stephen Senn."— Presentation transcript:
(C) Stephen Senn 20041 Change from baseline or analysis of covariance?: Lord's Paradox and other matters. Stephen Senn
(C) Stephen Senn 20042 Outline Adjustment in Randomised Clinical Trials –The argument for ANCOVA Lord’s Paradox –ANCOVA versus simple analysis of change scores (SACS) Observational studies –The argument against ANCOVA Resolution? –Why ANCOVA although not perfect may be best after all
(C) Stephen Senn 20043 SACS and ANCOVA A simple randomised clinical trial in which there are two treatment groups and only two measurements per patient: a baseline measurement, X and an outcome measurement, Y. Popular choices of outcome measure are 1) raw outcomes Y 2) change score d = Y - X 3) covariance adjusted outcomes Y - X. (where is chosen appropriately) NB As Laird (Am Stat., 37, 329-330, 1983) has shown, covariate adjusted change scores are the same as 3)
(C) Stephen Senn 20044 The Estimators Associated with the Measures If subscript t stands for treatment and c for control we have: 1) and 2) are just special cases of 3). If is chosen to be the regression of Y on X, then 3) corresponds to analysis of covariance. 1) 2) 3)
(C) Stephen Senn 20045 Warning These three measures, measure the same thing No question of choosing between them on the basis of clinical relevance Can only choose between them on the basis either of variance, or statistical philosophy ANCOVA may generally be expected to have the lowest variance Baseline is irrelevant to the definition of the treatment effect.
7 ANCOVA and Baseline by Treatment Interaction It is often stated that ANCOVA relies on the parallelism assumption. This is not true. If the effect of treatment varies with baseline it varies whether or not ANCOVA is used. ANCOVA is a first approximation and better than either doing nothing or using change scores.
(C) Stephen Senn 20048 Not to use ANCOVA, because you fear parallelism may not apply, is like saying crossing the channel in a rowing boat is dangerous I prefer to swim”.
(C) Stephen Senn 20049 Dichotomania Obsessive compulsive disorder –Cochrane Collaboration has a galloping case Numbers Needed to Treat should have been strangled at birth Division of patients into sheep and goats –Ignoring existence of geep and shoats Use of difference from baseline –Sin number one Destruction of information Arbitrary division into responders non-responders –Sin number two Further destruction of information Unjustified causal interpretation
(C) Stephen Senn 200410 A Red Herring It is sometimes claimed that measurement error invalidates ANCOVA –The reason is that if baseline is measured with error the regression of outcome on baseline is attenuated However this claim is incorrect ANCOVA is still valid –The reason is that it is appropriate to correct for an observed imbalance using an observed regression
(C) Stephen Senn 200411 Counter-Claims There is a significant minority of papers arguing against ANCOVA as a means of dealing with bias –E.g. Liang and Zeger (2000), Sankyha, Samuelson (1986), American Statistician The variance claims are accepted Claims are made that ANCOVA is biased unless there is balance at baseline
(C) Stephen Senn 200412 Justification of the Counter-Claim
(C) Stephen Senn 200413 Lord’s Paradox Lord, F.M. (1967) “ A paradox in the interpretation of group comparisons”, Psychological Bulletin, 68, 304-305. “A large university is interested in investigating the effects on the students of the diet provided in the university dining halls….Various types of data are gathered. In particular the weight of each student at the time of his arrival in September and his weight in the following June are recorded”
(C) Stephen Senn 200414 Two Statisticians Statistician One Calculates difference in weight for each hall Finds non-significant difference in each case (Also no difference between halls) Statistician Two Adjust for initial weight Finds significant hall effect Concludes difference between halls
(C) Stephen Senn 200415 A Simulated Example Starting and final weights for two groups of students Males and females 300 In each group Analysis illustrated with S Plus
(C) Stephen Senn 200419 Statistician Two’s analysis Call: lm(formula = Y ~ X + sex) Coefficients: Value Std. Error t value Pr(>|t|) (Intercept) 12.4987 1.5135 8.2584 0.0000 X 0.8087 0.0232 34.9156 0.0000 sex 1.9816 0.2833 6.9945 0.0000
(C) Stephen Senn 200420 What People Usually Conclude Where baseline values are not expected to be equal between groups ANCOVA can mislead Therefore even though SACS will have a higher variance it should be preferred under such circumstances since it is obviously unbiased
(C) Stephen Senn 200421 A Counter Counter-Example Suppose we design a bizarre clinical trial Only person with diastolic blood pressure at baseline equal to 95mmHg or 105mmHg may enter In the first stratum they are randomised 3 to 1 and in the second 1 to 3 Situation as follows
(C) Stephen Senn 200422 A Stupid Trial Numbers of Patients by dbp and Treatment Treatment ABTotal Baseline diastolic blood pressure 95mm Hg300100400 105 mmHg100300400 Total400 800
(C) Stephen Senn 200423 Approach to Analysis Stratify by baseline dbp Produce treatment estimate for each stratum Overall estimate is average of the two Stratification deals with the imbalance
(C) Stephen Senn 200424 An Equivalent Approach Create dummy variable stratum S = -1 if baseline dbp, X = 95mmHg S = 1 if baseline dbp, X =105 mmHg Regress dbp at outcome, Y, on treatment indicator T and on stratum indicator S Estimate will be same as by stratification If you want variance estimate to be exactly the same you need to include interaction also
(C) Stephen Senn 200425 An Equivalent Equivalent Approach Regress Y on T and X rather than on T and S –This is called ANCOVA! Note that S= (X-100)/5 Hence this approach is equivalent to the previous one, which is equivalent to stratification, which is unbiased On the other hand SACS is biased Hence we have produced a counter-example
(C) Stephen Senn 200426 Conclusion Contrary to what is often claimed there are cases where ANCOVA is unbiased but SACS is biased. No simple statement of the form “ANCOVA is more efficient but SACS is unbiased” is possible. In fact it is very difficult to imagine cases where SACS is the preferred analysis
(C) Stephen Senn 200427 Lord’s Paradox Revisited Statistician one assumes that in the absence of any differential treatment effect the two groups despite different baselines would show equivalent changes Statistician two assumes that in the absence of any differential treatment effect the change of the groups as a whole is the same as the change within groups Both of these causal assumptions are untestable
(C) Stephen Senn 200428 However It is easy to design trials for which –ANCOVA is unbiased –SACS is biased –A causal interpretation can be given It is rather difficult to design trials for which –SACS is unbiased –ANCOVA is biased –A causal interpretation can be given
(C) Stephen Senn 200429 The Necessary Condition for ANCOVA to be Unbiased Or in everyday language that the bias in the raw comparison at outcome should be times the bias at baseline where is the individual regression effect
(C) Stephen Senn 200430 Cut-off Designs Trochim and Capelleri have suggested that in many clinical trials randomisation will be unethical because some patients by the nature of their illness may be unwilling to assume the risks associated with an experimental treatment. They propose a class of designs called “cut-off” designs in which some patients are assigned to treatment in a deterministic manner on the basis of baseline values. The position, for example, might be as given in the diagram below.
(C) Stephen Senn 200431 Randomise to standard or experimental Experimental treatment only Standard treatment only Severe hypertensionModerate hypertension Mild hypertension A Cut-off Design in Hypertension
(C) Stephen Senn 200432 Cut off Designs Provided that the relationship between baseline and outcome is linear ANCOVA is valid Cut off designs are thus a wide class of design for which ANCOVA is unbiased SACS will be biased Thus we have more counterexamples to the claims of Liang and Zeger
(C) Stephen Senn 200433 A Challenge Can you design a trial for which –SACS is unbiased –ANCOVA is biased –A causal interpretation can be given?
(C) Stephen Senn 200434 Some Schemes That won’t Work Select patient according to true baseline values –Not possible in practice since not known –Still won’t work since correlation of true values is not 1 Select patients according to average of values at baseline and outcome –You need a crystal ball Select according to some other value –ANCOVA will be biased but so will SACS Select on binary covariate –Either this is permanent (e.g. sex), in which case causal inference doubtful –Or it varies over time in which case there will be a regression
(C) Stephen Senn 200435 Conclusion In RCTs ANCOVA is the appropriate way to use baseline information –SACS, responder analysis, NNTs all wasteful A hallmark of second rate analysis In observational studies things are more complex –ANCOVA may not be perfect but it may be the best you can do