Presentation on theme: "(C)Stephen Senn1 Lost Opportunities for Design Theory in Drug Development Stephen Senn."— Presentation transcript:
(C)Stephen Senn1 Lost Opportunities for Design Theory in Drug Development Stephen Senn
(C)Stephen Senn2 Basic Thesis Design theory has great potential in drug development But this potential is unrealised Those working in so-called optimal design are so ignorant of application realities that where their influence is not zero it is harmful On the other hand the understanding of design theory by biostatisticians is pitifully inadequate We must cooperate properly to cure this parlous state of affairs
(C)Stephen Senn3 Outline Quick tutorial on cross-over trials I shall then give two introductory examples of nonsense –By leading design theoreticians –By leading biostatisticians I shall then consider design nonsense further Some conclusions After lunch a case-study
(C)Stephen Senn4 Warning I am a biostatistician We are used to thinking of data matrices with rows as subjects and columns as measurements That means that we write sequences for designs with rows representing subjects and columns representing periods
(C)Stephen Senn5 Cross-over Trials Definition: A cross-over trial is one in which subjects are given sequences with the object of studying differences between individual treatments.
(C)Stephen Senn6 An Example of an AB/BA cross-over in asthma
(C)Stephen Senn7 An Example from Rheumatism: 2 doses of diclofenac and placebo
(C)Stephen Senn8 Carry-over Definition: Carry-over is the persistence (whether physically or in terms of effect) of a treatment applied in one period in a subsequent period of treatment. If carry-over applies in a cross-over trial we shall, at some stage, observe the simultaneous effects of two or more treatments on given patients. We may, however, not be aware that this is what we are observing and this ignorance may lead us to make errors in interpretation.
(C)Stephen Senn9 Simple Carry-over Carry-over lasts for exactly one period It depends only on the engendering treatment and is unmodified by the perturbed treatment There is a huge literature proposing optimal designs for this model There is no empirical evidence that any of this has been useful
(C)Stephen Senn10 Three Period Bioequivalence Designs Three formulation designs in six sequences common. Subjects randomised in equal numbers to six possible sequences. – For example, 18 subjects, three on each of the sequences ABC, ACB, BAC, BCA, CAB, CBA. – A = test formulation under fasting conditions, – B = test formulation under fed conditions – C = reference formulation under fed conditions.
(C)Stephen Senn11 Weights for the Three Period Design: not Adjusting for Carry-over
(C)Stephen Senn12 Properties of these weights Sum 0 in any column, –eliminates the period effect. Sum 0 in any row –eliminates patient effect Sum 0 over cells labelled A –A has no part in definition of contrast Sum to 1 over the cells labelled B and to -1 over the cells labelled C –Estimate contrast B-C
(C)Stephen Senn13 Weights for the Three Period Design: Adjusting for Carry-over
(C)Stephen Senn14 Weights for the Three Period Design: Adjusting for Carry-over
(C)Stephen Senn15 Properties of These Weights As before –Estimates B-C contrast –Eliminates, period and patient effect –Eliminates A Sum to zero over cells labelled a,b, and c –Eliminate simple carry-over
(C)Stephen Senn16 Have We Got Something for Nothing? Sum of squares weights of first scheme is 1/3 (or 4/12) Sum of squares of weights of second scheme is 5/12 Given independent homoscedastic within- patient errors, there is thus a 25% increase in variance Penalty for adjusting is loss of efficiency
(C)Stephen Senn17 First Example Some Design Theory Nonsense John, J. A., Russell, K. G., and Whitaker, D. (2004), "Crossover: An Algorithm for the Construction of Efficient Cross-over Designs," Statistics in Medicine, 23, 2645 - 2658. A cross-over experiment involves the application of sequences of treatments to several subjects over a number of time periods. It is thought that the observation made on each subject at the end of a time period may depend on the direct effect of the treatment applied in the current period, and the carry-over effects of the treatments applied in one or more previous periods. Various models have been proposed to explain the nature of the carry-over effects. An experimental design that is optimal under one model may not be optimal if a different model is the appropriate one. In this paper an algorithm is described to construct efficient cross-over designs for a range of models that involve the direct effects of the treatments and various functions of their carry-over effects. The effectiveness and flexibility of the algorithm are demonstrated by assessing its performance against numerous designs and models given in the literature.
(C)Stephen Senn18 Sometimes in a clinical trial it may be necessary to modify or extend an ongoing trial. For example, suppose that after a few periods have been completed one of the treatments is dropped from the trial. It will then be necessary to re-allocate the other treatments to the remaining periods of the trial. (John et al, 2004 p. 2653) Jones and Donev  also consider augmenting a design to account for the removal of a treatment. The initial trial to compare four treatments A, B, C, and D in five periods using four groups of subjects used a Williams square  with the fourth period repeated. After the first two periods had been completed it was decided to drop treatment D from the remainder of the trial. Whats wrong here?
(C)Stephen Senn20 The reality..in a single-dose cross-over trial in asthma in 12 patients reported by Palmqvist et al.  patients were treated in the first period of a cross-over on dates ranging from 5 May to 12 November 1987 . They were treated in a second period on dates ranging from 18 May to 26 November 1987. Eleven of the patients had completed period two of treatment before the 12th patient was recruited. (Senn, 2005, p3675.) A basic fact of clinical trials You treat patients when they fall ill
(C)Stephen Senn22 Conclusion Multi-dose trials real scope for design theory. These will employ active wash-out Design problem is trade-off between exploiting correlation and eliminating carry-over. Short vs long active wash-out periods
(C)Stephen Senn23 Second Example Some Biostatistics Nonsense Chow, S. C., and Liu, J. P. (2000), Design and Analysis of Bioavailability and Bioequivalence Studies (2nd ed.), New York: Marcel Dekker. Have several discussions of efficiency of designs in their book which are completely beside the point. They compare designs in terms of residual degrees of freedom! They conclude that Balaams design, which uses sequences TR/RT/TT/RR Is similar in efficiency to the more conventional TR/RT design. They write The degrees of freedom for the intrasubject residuals for the 2 × 2 and 4 × 2 design are 22 and 21 respectively. Therefore there is little difference in testing power. This is nonsense
(C)Stephen Senn24 Second Example Some Biostatistics Nonsense. Design Source 2×24×22×32×44×4 Between 23 Seq 1311 Res 222022 Within 24 4872 Period 1123 Form 1111 Carry 111 Res 22214467 Total 47 7195
(C)Stephen Senn25 What is wrong 1. Its not correct design theory As any design expert knows residual degrees of freedom are (nearly) irrelevant to efficiency It is the impact of adjustment on the degree of orthogonality of the design matrix that is important
(C)Stephen Senn26 What is wrong 2. Its not realistic biostatistics In fact as any biostatistician who has had to think about it will know from a practical point of view far from being optimal Balaams design is simply inadmissible The reasons is that only half of the resources are devoted to actually measuring the treatment The rest are devoted to providing an adjustment for a form of carry-over that is itself implausible
(C)Stephen Senn27 Allocation of patients for two designs SequenceAB/BABalaam ABn/2n/4 BAn/2n/4 AA0n/4 BB0n/4
(C)Stephen Senn28 Investigation of the real efficiency of Balaams design
(C)Stephen Senn29 Five Reasons why the Simple Carry-over Model is not Useful If it applies then the investigator can design a trial which eliminates it. (double the periods) Implausible given pk/pd theory. (obvious) Leads to inefficient estimators. (see investigation to follow) Can lead to poor designs. (ditto) The models which incorporate it are self- contradicting. (example: factorial X-overs)
(C)Stephen Senn30 Dose response: the pharmacokineticists version
(C)Stephen Senn31 Dose Response: The Statisticians Version Dose Response This is what the simple carry- over model implies
(C)Stephen Senn32 The Models Which use Simple Carry-over are Inconsistent Consider a factorial cross-over in four periods comparing A, B and the combination of A and B to placebo. We can represent the four treatments by: **, A*, *B and AB. Suppose we consider a patient who has received the sequence AB ** A* *B. A standard parameterisation for treatment and carry-over would be as in the following table.
(C)Stephen Senn33 Paramaterisation of a factorial cross-over Period
(C)Stephen Senn34 The Rhinoceros The rhinoceros has a kind heart, if you doubt it heres the proof That thing on his nose is for taking stones out of a horses hoof He seldom ever meets a horse, it is this that makes him sad When he does, then it hasnt a stone in its hoof But he would, if he did and it had Flanders and Swann
(C)Stephen Senn35 The Phoenix Bioequivalence Trials Analysed by DAngelo & Potvin 20 drug classes 1989-1999 12 or more subjects 96 three period designs 324 two period designs
(C)Stephen Senn36 Three Treatment Designs P-Values for Carry-Over
(C)Stephen Senn38 Test of Uniformity of P-Values
(C)Stephen Senn39 Conclusions Distribution of P-values uniform –no evidence of carry-over Carry-over a priori implausible –presence testable by assay No point is testing for it –leads to bias Or adjusting for it –increased variance
(C)Stephen Senn40 Do Bayesians do Better? In principle the Bayesian approach ought to allow us to be more flexible about nuisance parameters such as carry-over However, the Bayesian track record is not impressive here Realistic models have not been employed
(C)Stephen Senn41 Cross-over trial in Eneuresis Two treatment periods of 14 days each Treatment effect significant if carry-over not fitted 2.037 ( 0.768, 3.306) Treatment effect not significant if carry-over fitted 0.451 (-2.272, 3.174) 1. Hills, M, Armitage, P. The two-period cross-over clinical trial, British Journal of Clinical Pharmacology 1979; 8: 7-20.
(C)Stephen Senn42 Identical uninformative prior placed on carry-over as for treatment NB Parameterisation here means that values of need to be doubled to compare to conventional contrasts
(C)Stephen Senn43 Identical Priors for Treatment and Carryover? Patients treated repeatedly during trial Fourteen day treatment period Average time to last treatment plausibly 4 hours Average time to previous treatment seven days Saying that it is just as likely that carry-over could be greater than treatment is not coherent In any case the two cannot be independent Is negative carry-over as likely as positive carry- over?
(C)Stephen Senn44 So What are Acceptable Models for Carry-over? Ignoring carry-over altogether (not allowing for it because one believes one has taken adequate steps to eliminate it) –This is always a reasonable strategy Using an integrated pharmacokinetic pharmacodynamic model (Sheiner et al, 1991) –This may work for dose-finding trials –Very difficult to implement where more than one molecule is involved
(C)Stephen Senn45 The Sheiner model PD dose response PK model for dose- concentration as a consequence of previous dosing history Steady state concentration for patient i in period l
(C)Stephen Senn49 The difference between mathematical and applied statistics is that the former is full of lemmas whereas the latter is full of dilemmas
(C)Stephen Senn50 Advice for Design-Theoreticians Resist the temptation to give advice if you are unfamiliar with the application area Seek collaborators Ground your models in pharmacology Remember that the goal is good medicine not elegant mathematics Dont defend the indefensible
(C)Stephen Senn51 Advice for biostatisticians Remember that design theoreticians have many powerful results Its just conceivable that some of them may even be useful
(C)Stephen Senn53 References 1.Senn, S.J., Is the 'simple carry-over' model useful? [published erratum appears in Statistics in Medicine 1992 Sep 15;11(12):1619]. Statistics in Medicine, 1992. 11(6): p. 715-26. 2.Senn, S.J., The AB/BA cross-over: how to perform the two-stage analysis if you can't be persuaded that you shouldn't., in Liber Amicorum Roel van Strik, B. Hansen and M. de Ridder, Editors. 1996, Erasmus University: Rotterdam. p. 93-100. 3.Senn, S.J., Cross-over Trials in Clinical Research. Second ed. 2002, Chichester: Wiley. 4.Senn, S.J., Statistical Issues in Drug Development. Statistics in Practice, ed. V. Barnett. 2007, Chichester: John Wiley.