Presentation is loading. Please wait.

Presentation is loading. Please wait.

Change from baseline or analysis of covariance

Similar presentations

Presentation on theme: "Change from baseline or analysis of covariance"— Presentation transcript:

1 Change from baseline or analysis of covariance
Change from baseline or analysis of covariance?: Lord's Paradox and other matters. Stephen Senn Change from baseline or analysis of covariance?: Lord's Paradox and other matters. In longitudinal studies it is generally agreed that analysis of covariance (ANCOVA) fitting baselines provides treatment estimates with lower variances than does a simple analysis of change scores (SACS). However, it also sometimes claimed that unless the means at baseline of groups being compared are equal, then such ANCOVA adjusted estimates are biased whereas SACS estimates are not. This would seem to imply that for cases where "Lord's paradox" applies, that is to say where SACS and ANCOVA may be expected to give different answers, the former should be preferred to the latter. In this talk I shall disagree with this point of view and show that cases where SACS is unbiased and ANCOVA is not are extremely difficult to construct in practice if a causal interpretation can still be given to the "treatment estimate" but that the reverse is not the case. This suggests that cases where SACS can be a good approach to analysis are few and far between. (C) Stephen Senn 2004

2 Outline Adjustment in Randomised Clinical Trials Lord’s Paradox
The argument for ANCOVA Lord’s Paradox ANCOVA versus simple analysis of change scores (SACS) Observational studies The argument against ANCOVA Resolution? Why ANCOVA although not perfect may be best after all (C) Stephen Senn 2004

3 SACS and ANCOVA A simple randomised clinical trial in which there are two treatment groups and only two measurements per patient: a baseline measurement, X and an outcome measurement, Y. Popular choices of outcome measure are 1) raw outcomes Y 2) change score d = Y - X 3) covariance adjusted outcomes Y - X. (where  is chosen appropriately) NB As Laird (Am Stat., 37, , 1983) has shown, covariate adjusted change scores are the same as 3) (C) Stephen Senn 2004

4 The Estimators Associated with the Measures
If subscript t stands for treatment and c for control we have: 1) 2) 3) 1) and 2) are just special cases of 3). If  is chosen to be the regression of Y on X, then 3) corresponds to analysis of covariance. (C) Stephen Senn 2004

5 Warning These three measures, measure the same thing
No question of choosing between them on the basis of clinical relevance Can only choose between them on the basis either of variance, or statistical philosophy ANCOVA may generally be expected to have the lowest variance Baseline is irrelevant to the definition of the treatment effect. (C) Stephen Senn 2004

6 (C) Stephen Senn 2004

7 ANCOVA and Baseline by Treatment Interaction
It is often stated that ANCOVA relies on the parallelism assumption. This is not true. If the effect of treatment varies with baseline it varies whether or not ANCOVA is used. ANCOVA is a first approximation and better than either doing nothing or using change scores. (C) Stephen Senn 2004

8 Not to use ANCOVA, because you fear parallelism may not apply, is like saying crossing the channel in a rowing boat is dangerous I prefer to swim”. (C) Stephen Senn 2004

9 Dichotomania Obsessive compulsive disorder
Cochrane Collaboration has a galloping case Numbers Needed to Treat should have been strangled at birth Division of patients into sheep and goats Ignoring existence of geep and shoats Use of difference from baseline Sin number one Destruction of information Arbitrary division into responders non-responders Sin number two Further destruction of information Unjustified causal interpretation (C) Stephen Senn 2004

10 A Red Herring It is sometimes claimed that measurement error invalidates ANCOVA The reason is that if baseline is measured with error the regression of outcome on baseline is attenuated However this claim is incorrect ANCOVA is still valid The reason is that it is appropriate to correct for an observed imbalance using an observed regression (C) Stephen Senn 2004

11 Counter-Claims There is a significant minority of papers arguing against ANCOVA as a means of dealing with bias E.g. Liang and Zeger (2000), Sankyha, Samuelson (1986), American Statistician The variance claims are accepted Claims are made that ANCOVA is biased unless there is balance at baseline (C) Stephen Senn 2004

12 Justification of the Counter-Claim
(C) Stephen Senn 2004

13 Lord’s Paradox Lord, F.M. (1967) “ A paradox in the interpretation of group comparisons”, Psychological Bulletin, 68, “A large university is interested in investigating the effects on the students of the diet provided in the university dining halls….Various types of data are gathered. In particular the weight of each student at the time of his arrival in September and his weight in the following June are recorded” (C) Stephen Senn 2004

14 Two Statisticians Statistician One
Calculates difference in weight for each hall Finds non-significant difference in each case (Also no difference between halls) Statistician Two Adjust for initial weight Finds significant hall effect Concludes difference between halls (C) Stephen Senn 2004

15 A Simulated Example Starting and final weights for two groups of students Males and females 300 In each group Analysis illustrated with S Plus (C) Stephen Senn 2004

16 (C) Stephen Senn 2004

17 Statistician One’s Analysis
Paired t-Test data: Y.males and X.males t = 0.662, df = 299, p-value = data: Y.females and X.females t = , df = 299, p-value = Standard Two-Sample t-Test data: diff.males and diff.females t = , df = 598, p-value = (C) Stephen Senn 2004

18 (C) Stephen Senn 2004

19 Statistician Two’s analysis
Call: lm(formula = Y ~ X + sex) Coefficients: Value Std. Error t value Pr(>|t|) (Intercept) X sex (C) Stephen Senn 2004

20 What People Usually Conclude
Where baseline values are not expected to be equal between groups ANCOVA can mislead Therefore even though SACS will have a higher variance it should be preferred under such circumstances since it is obviously unbiased (C) Stephen Senn 2004

21 A Counter Counter-Example
Suppose we design a bizarre clinical trial Only person with diastolic blood pressure at baseline equal to 95mmHg or 105mmHg may enter In the first stratum they are randomised 3 to 1 and in the second 1 to 3 Situation as follows (C) Stephen Senn 2004

22 A Stupid Trial Numbers of Patients by dbp and Treatment
Total Baseline diastolic blood pressure 95mm Hg 300 100 400 105 mmHg 800 (C) Stephen Senn 2004

23 Approach to Analysis Stratify by baseline dbp
Produce treatment estimate for each stratum Overall estimate is average of the two Stratification deals with the imbalance (C) Stephen Senn 2004

24 An Equivalent Approach
Create dummy variable stratum S = -1 if baseline dbp, X = 95mmHg S = 1 if baseline dbp, X =105 mmHg Regress dbp at outcome, Y, on treatment indicator T and on stratum indicator S Estimate will be same as by stratification If you want variance estimate to be exactly the same you need to include interaction also (C) Stephen Senn 2004

25 An Equivalent Equivalent Approach
Regress Y on T and X rather than on T and S This is called ANCOVA! Note that S= (X-100)/5 Hence this approach is equivalent to the previous one, which is equivalent to stratification, which is unbiased On the other hand SACS is biased Hence we have produced a counter-example (C) Stephen Senn 2004

26 Conclusion Contrary to what is often claimed there are cases where ANCOVA is unbiased but SACS is biased. No simple statement of the form “ANCOVA is more efficient but SACS is unbiased” is possible. In fact it is very difficult to imagine cases where SACS is the preferred analysis (C) Stephen Senn 2004

27 Lord’s Paradox Revisited
Statistician one assumes that in the absence of any differential treatment effect the two groups despite different baselines would show equivalent changes Statistician two assumes that in the absence of any differential treatment effect the change of the groups as a whole is the same as the change within groups Both of these causal assumptions are untestable (C) Stephen Senn 2004

28 However It is easy to design trials for which
ANCOVA is unbiased SACS is biased A causal interpretation can be given It is rather difficult to design trials for which SACS is unbiased ANCOVA is biased A causal interpretation can be given (C) Stephen Senn 2004

29 The Necessary Condition for ANCOVA to be Unbiased
Or in everyday language that the bias in the raw comparison at outcome should be  times the bias at baseline where  is the individual regression effect (C) Stephen Senn 2004

30 Cut-off Designs Trochim and Capelleri have suggested that in many clinical trials randomisation will be unethical because some patients by the nature of their illness may be unwilling to assume the risks associated with an experimental treatment. They propose a class of designs called “cut-off” designs in which some patients are assigned to treatment in a deterministic manner on the basis of baseline values. The position, for example, might be as given in the diagram below. (C) Stephen Senn 2004

31 A Cut-off Design in Hypertension
Randomise to standard or experimental Experimental treatment only Standard treatment only Mild hypertension Moderate hypertension Severe hypertension (C) Stephen Senn 2004

32 Cut off Designs Provided that the relationship between baseline and outcome is linear ANCOVA is valid Cut off designs are thus a wide class of design for which ANCOVA is unbiased SACS will be biased Thus we have more counterexamples to the claims of Liang and Zeger (C) Stephen Senn 2004

33 A Challenge Can you design a trial for which SACS is unbiased
ANCOVA is biased A causal interpretation can be given? (C) Stephen Senn 2004

34 Some Schemes That won’t Work
Select patient according to true baseline values Not possible in practice since not known Still won’t work since correlation of true values is not 1 Select patients according to average of values at baseline and outcome You need a crystal ball Select according to some other value ANCOVA will be biased but so will SACS Select on binary covariate Either this is permanent (e.g. sex), in which case causal inference doubtful Or it varies over time in which case there will be a regression (C) Stephen Senn 2004

35 Conclusion In RCTs ANCOVA is the appropriate way to use baseline information SACS, responder analysis, NNTs all wasteful A hallmark of second rate analysis In observational studies things are more complex ANCOVA may not be perfect but it may be the best you can do (C) Stephen Senn 2004

36 Here there be tygers! (C) Stephen Senn 2004

Download ppt "Change from baseline or analysis of covariance"

Similar presentations

Ads by Google