Presentation is loading. Please wait.

Presentation is loading. Please wait.

(C) Stephen Senn 20041 Change from baseline or analysis of covariance?: Lord's Paradox and other matters. Stephen Senn.

Similar presentations

Presentation on theme: "(C) Stephen Senn 20041 Change from baseline or analysis of covariance?: Lord's Paradox and other matters. Stephen Senn."— Presentation transcript:

1 (C) Stephen Senn Change from baseline or analysis of covariance?: Lord's Paradox and other matters. Stephen Senn

2 (C) Stephen Senn Outline Adjustment in Randomised Clinical Trials –The argument for ANCOVA Lord’s Paradox –ANCOVA versus simple analysis of change scores (SACS) Observational studies –The argument against ANCOVA Resolution? –Why ANCOVA although not perfect may be best after all

3 (C) Stephen Senn SACS and ANCOVA A simple randomised clinical trial in which there are two treatment groups and only two measurements per patient: a baseline measurement, X and an outcome measurement, Y. Popular choices of outcome measure are 1) raw outcomes Y 2) change score d = Y - X 3) covariance adjusted outcomes Y -  X. (where  is chosen appropriately) NB As Laird (Am Stat., 37, , 1983) has shown, covariate adjusted change scores are the same as 3)

4 (C) Stephen Senn The Estimators Associated with the Measures If subscript t stands for treatment and c for control we have: 1) and 2) are just special cases of 3). If  is chosen to be the regression of Y on X, then 3) corresponds to analysis of covariance. 1) 2) 3)

5 (C) Stephen Senn Warning These three measures, measure the same thing No question of choosing between them on the basis of clinical relevance Can only choose between them on the basis either of variance, or statistical philosophy ANCOVA may generally be expected to have the lowest variance Baseline is irrelevant to the definition of the treatment effect.

6 (C) Stephen Senn 20046

7 7 ANCOVA and Baseline by Treatment Interaction It is often stated that ANCOVA relies on the parallelism assumption. This is not true. If the effect of treatment varies with baseline it varies whether or not ANCOVA is used. ANCOVA is a first approximation and better than either doing nothing or using change scores.

8 (C) Stephen Senn Not to use ANCOVA, because you fear parallelism may not apply, is like saying crossing the channel in a rowing boat is dangerous I prefer to swim”.

9 (C) Stephen Senn Dichotomania Obsessive compulsive disorder –Cochrane Collaboration has a galloping case Numbers Needed to Treat should have been strangled at birth Division of patients into sheep and goats –Ignoring existence of geep and shoats Use of difference from baseline –Sin number one Destruction of information Arbitrary division into responders non-responders –Sin number two Further destruction of information Unjustified causal interpretation

10 (C) Stephen Senn A Red Herring It is sometimes claimed that measurement error invalidates ANCOVA –The reason is that if baseline is measured with error the regression of outcome on baseline is attenuated However this claim is incorrect ANCOVA is still valid –The reason is that it is appropriate to correct for an observed imbalance using an observed regression

11 (C) Stephen Senn Counter-Claims There is a significant minority of papers arguing against ANCOVA as a means of dealing with bias –E.g. Liang and Zeger (2000), Sankyha, Samuelson (1986), American Statistician The variance claims are accepted Claims are made that ANCOVA is biased unless there is balance at baseline

12 (C) Stephen Senn Justification of the Counter-Claim

13 (C) Stephen Senn Lord’s Paradox Lord, F.M. (1967) “ A paradox in the interpretation of group comparisons”, Psychological Bulletin, 68, “A large university is interested in investigating the effects on the students of the diet provided in the university dining halls….Various types of data are gathered. In particular the weight of each student at the time of his arrival in September and his weight in the following June are recorded”

14 (C) Stephen Senn Two Statisticians Statistician One Calculates difference in weight for each hall Finds non-significant difference in each case (Also no difference between halls) Statistician Two Adjust for initial weight Finds significant hall effect Concludes difference between halls

15 (C) Stephen Senn A Simulated Example Starting and final weights for two groups of students Males and females 300 In each group Analysis illustrated with S Plus

16 (C) Stephen Senn

17 (C) Stephen Senn Statistician One’s Analysis Paired t-Test data: Y.males and X.males t = 0.662, df = 299, p-value = Paired t-Test data: Y.females and X.females t = , df = 299, p-value = Standard Two-Sample t-Test data: diff.males and diff.females t = , df = 598, p-value =

18 (C) Stephen Senn

19 (C) Stephen Senn Statistician Two’s analysis Call: lm(formula = Y ~ X + sex) Coefficients: Value Std. Error t value Pr(>|t|) (Intercept) X sex

20 (C) Stephen Senn What People Usually Conclude Where baseline values are not expected to be equal between groups ANCOVA can mislead Therefore even though SACS will have a higher variance it should be preferred under such circumstances since it is obviously unbiased

21 (C) Stephen Senn A Counter Counter-Example Suppose we design a bizarre clinical trial Only person with diastolic blood pressure at baseline equal to 95mmHg or 105mmHg may enter In the first stratum they are randomised 3 to 1 and in the second 1 to 3 Situation as follows

22 (C) Stephen Senn A Stupid Trial Numbers of Patients by dbp and Treatment Treatment ABTotal Baseline diastolic blood pressure 95mm Hg mmHg Total

23 (C) Stephen Senn Approach to Analysis Stratify by baseline dbp Produce treatment estimate for each stratum Overall estimate is average of the two Stratification deals with the imbalance

24 (C) Stephen Senn An Equivalent Approach Create dummy variable stratum  S = -1 if baseline dbp, X = 95mmHg  S = 1 if baseline dbp, X =105 mmHg Regress dbp at outcome, Y, on treatment indicator T and on stratum indicator S Estimate will be same as by stratification If you want variance estimate to be exactly the same you need to include interaction also

25 (C) Stephen Senn An Equivalent Equivalent Approach Regress Y on T and X rather than on T and S –This is called ANCOVA! Note that S= (X-100)/5 Hence this approach is equivalent to the previous one, which is equivalent to stratification, which is unbiased On the other hand SACS is biased Hence we have produced a counter-example

26 (C) Stephen Senn Conclusion Contrary to what is often claimed there are cases where ANCOVA is unbiased but SACS is biased. No simple statement of the form “ANCOVA is more efficient but SACS is unbiased” is possible. In fact it is very difficult to imagine cases where SACS is the preferred analysis

27 (C) Stephen Senn Lord’s Paradox Revisited Statistician one assumes that in the absence of any differential treatment effect the two groups despite different baselines would show equivalent changes Statistician two assumes that in the absence of any differential treatment effect the change of the groups as a whole is the same as the change within groups Both of these causal assumptions are untestable

28 (C) Stephen Senn However It is easy to design trials for which –ANCOVA is unbiased –SACS is biased –A causal interpretation can be given It is rather difficult to design trials for which –SACS is unbiased –ANCOVA is biased –A causal interpretation can be given

29 (C) Stephen Senn The Necessary Condition for ANCOVA to be Unbiased Or in everyday language that the bias in the raw comparison at outcome should be  times the bias at baseline where  is the individual regression effect

30 (C) Stephen Senn Cut-off Designs Trochim and Capelleri have suggested that in many clinical trials randomisation will be unethical because some patients by the nature of their illness may be unwilling to assume the risks associated with an experimental treatment. They propose a class of designs called “cut-off” designs in which some patients are assigned to treatment in a deterministic manner on the basis of baseline values. The position, for example, might be as given in the diagram below.

31 (C) Stephen Senn Randomise to standard or experimental Experimental treatment only Standard treatment only Severe hypertensionModerate hypertension Mild hypertension A Cut-off Design in Hypertension

32 (C) Stephen Senn Cut off Designs Provided that the relationship between baseline and outcome is linear ANCOVA is valid Cut off designs are thus a wide class of design for which ANCOVA is unbiased SACS will be biased Thus we have more counterexamples to the claims of Liang and Zeger

33 (C) Stephen Senn A Challenge Can you design a trial for which –SACS is unbiased –ANCOVA is biased –A causal interpretation can be given?

34 (C) Stephen Senn Some Schemes That won’t Work Select patient according to true baseline values –Not possible in practice since not known –Still won’t work since correlation of true values is not 1 Select patients according to average of values at baseline and outcome –You need a crystal ball Select according to some other value –ANCOVA will be biased but so will SACS Select on binary covariate –Either this is permanent (e.g. sex), in which case causal inference doubtful –Or it varies over time in which case there will be a regression

35 (C) Stephen Senn Conclusion In RCTs ANCOVA is the appropriate way to use baseline information –SACS, responder analysis, NNTs all wasteful A hallmark of second rate analysis In observational studies things are more complex –ANCOVA may not be perfect but it may be the best you can do

36 (C) Stephen Senn Here there be tygers!

Download ppt "(C) Stephen Senn 20041 Change from baseline or analysis of covariance?: Lord's Paradox and other matters. Stephen Senn."

Similar presentations

Ads by Google