1. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 31 Lecture 3 Sequential designs 3.1 A multi-stage (or group sequential) design 3.2 A general setting for sequential analysis 3.3 Design using the SAS function SEQ 3.4 Choice of design 3.5 Final analysis

2. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 32 Z n 3.1 A multi-stage (or group sequential) design PROCEED : E > C ABANDON E Continue if Z i  ( i, u i )

3. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 33 Design calculation Once more, we wish to satisfy the 2 equations: P( PROCEED ;  = 0) =  and P( PROCEED ;  =  R ) = 1   For k looks at the data, we have 3k  1 unknowns: n i, i, u i (but no k ) It is usual to specify r 2,..., r k and put n i = r i n 1, i = 2,..., k Then define functions i = i (a), u i = u i (a) This leaves two unknowns: n 1 and a

4. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 34 Examples (Put r 1 = 1) (1) i =  a, u i = a (Pocock, 1977) (2) i =  a/  r i, u i = a/  r i (O’Brien & Fleming, 1979) (3) i =  a{1 – 3(r i /r k )}/  r i, u i = a{1 + (r i /r k )}/  r i The triangular test (Whitehead, 1997)

5. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 35 In Lecture 2, patients were randomised equally between treatments E and C and their responses were normally distributed: for the h th patient on treatment j, the response was Y hj Y hj ~ N(  j,  2 ), h = 1, 2,... ; j = E, C The variance  2 was treated as known, and the advantage of E over C was taken to be  =  E   C 3.2 A general setting for sequential analysis

6. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 36 At the i th interim analysis, the test statistic Z i is computed where and n i is the total number of responses Now put then

7. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 37 Furthermore and so (B i+1 – B i ) and B i are independent for all i The process {B i } is said to have independent increments The theory of sequential tests (including two stage tests) applies to all sequences of test statistics {B i } where B i has independent increments and B i ~ N(  V i, V i ) for all i We plot Z i = B i /  V i against V i, or B i against V i

8. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 38 Binary responses 1:1 randomisation between E and C Patients SUCCEED or FAIL P( SUCCESS ) = p E and p C on E and C respectively The advantage of E over C is expressed as the log-odds ratio:

9. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 39 Binary data at the i th interim analysis: Put and then B i has independent increments and B i ~ N(  V i, V i ) for all i, approximately, when the n i are large and  is small treatmentECOverall SUCCEED S Ei S Ci SiSi FAIL F Ei F Ci FiFi totaln Ei n Ci nini

10. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 310 Note that:   when n i is large, where

11. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 311 Survival responses 1:1 randomisation between E and C Analyse the time from randomisation to failure hazard functions: h E (t) and h C (t) survival functions: S E (t) and S C (t) The advantage of E over C is expressed as the log-hazard ratio:

12. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 312 Put B i = the logrank statistic and V i = var(B i   = 0) then B i has independent increments and B i ~ N(  V i, V i ) for all i, approximately, when the n i are large and  is small  when n i is large and  is small, then where e i is the number of events occurring by the i th interim analysis

13. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 313 Some general comments  In each of these cases B i = efficient score = d (0)/d  V i = Fisher’s information =  d 2 (0)/d   where denotes the log of the likelihood, the profile likelihood or the partial likelihood  In my book and papers, I write Z i for B i !  Can stratify or allow for covariates  Can also use

14. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 314 3.3 Design using the SAS function SEQ SEQ is a function within the SAS procedure PROC IML  Interactive Matrix Language It is called as follows: call seq(prob, domain) tscale = tscale; Suppose that the design is defined as in Section 3.1 with V i, i, u i, i = 1,..., k Now we define in terms of V i rather than n i SAS allows a value for k, which can be set to u k – 0.000001 if desired

15. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 315 Let r i = V i /V 1, i = 1,..., k, (so that r 1 = 1) then domain is the 2  k matrix tscale is the (k – 1) vector prob is the output, in the form of a 3  k matrix where q i = P(reach i th interim with Z i  i   = 0 ) p i = P(reach i th interim with Z i  u i   = 0 ) h i = P(reach i th interim   = 0 )

16. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 316 Put pu i = h i – p i = P(stop on upper boundary at i th interim) We want the design to satisfy pu 1 (0) +... + pu k (0) = P( PROCEED ;  = 0) =  If all of the stopping limits i, u i, i = 1,..., k are functions of a single parameter a, and the ratios r i = V i /V 1 are known, then we can solve for a Note p i = q i = P(stop on lower boundary at i th interim)

17. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 317 Having found a, and therefore i and u i, we know that the required design will continue as long as Z i  ( i, u i ) This is equivalent to: continue as long as B i  ( i  V i, u i  V i ) and also equivalent to: continue as long as B i –  V i  ( i  V i –  V i, u i  V i –  V i ) and to: continue as long as

18. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 318 As the left-hand side above is standard normal, to find properties for general , enter the domain as: We also want the design to satisfy pu 1 (  R ) +... + pu k (  R ) = P( PROCEED ;  =  R ) = 1   So, enter the domain as above, with  =  R, and search for the value of V 1 that makes this power equal to 1 

19. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 319 3.4 Choice of design Compare through plots of B against V, (boundaries *, u*)  easier to understand as E(B) =  V, so that the plot tends to move as a straight line with slope   as E(Z) =  V, plots of Z against V tend to move as square root plots which are less easy to appreciate The lower boundary can correspond to either (a) ABANDON E, that is do not PROCEED or (b) STOP and conclude that E is worse than C

20. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 320 B V Pocock: i =  a, u i = a; i * =  a  V i, u i * = a  V i PROCEED: E > C STOP: E < C

21. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 321 B V O’Brien & Fleming: i =  a/  r i, u i = a/  r i ; i * =  a  V 1, u i * = a  V 1 PROCEED: E > C STOP: E < C

22. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 322 B V Triangular: i =  a{1 – 3(r i /r k )}/  r i, u i = a{1 + (r i /r k )}/  r i i * =  a  V 1 {1 – 3(r i /r k )}, u i * = a  V 1 {1 + (r i /r k )} PROCEED: E > C ABANDON E

23. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 323 Symmetric and asymmetric tests  Pocock’s and O’Brien and Fleming’s tests are examples of symmetric sequential tests that is: i =  u i for each i so that: P( STOP: E < C ;  =  R ) = 1    The triangular test is an example of an asymmetric sequential test that is: i   u i for each i so that: P( STOP: E < C ;  =  R )  1  

24. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 324 Use a symmetric test if it is just as important to prove harm as to establish benefit:  when comparing two existing treatments Use an asymmetric test if it is not important to prove harm, lack of benefit will lead to abandoning E  when comparing a new or expensive or hazardous treatment with placebo or standard All fixed sample tests are symmetric

25. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 325 Expected sample size Can compare tests in terms of E(V*;  ), where V* is the amount of information at termination From the SAS function SEQ, we have Pocock: E(V*;  ) is small when  is large, but is bigger when   0, where it substantially exceeds V fixed O’B&F: E(V*;  ) is small when  is very large, bigger when   0, but never much bigger than V fixed

26. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 326 The triangular test (approximately) minimises amongst tests satisfying the power requirement The triangular test keeps E(V*;  ) small when  < 0, in exchange for accepting a low power for claiming that E is significantly harmful

27. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 327 B V 3.5 Final analysis PROCEED: E > C ABANDON E x x x One-sided p-value is probability of reaching the pink area if  = 0

28. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 328  p-value can be computed using the SAS function SEQ  the p-value function is p(  ) = P(reaching pink zone   ) 95% confidence interval is (  L,  U ) where p(  L ) = 0.025 and p(  U ) = 0.975 median unbiased estimate is  M where p(  M ) = 0.5  these analyses differ from analyses based on the final dataset that ignore the sequential design  ignoring the sequential design is invalid  when stopping to PROCEED is possible, such invalid analyses might overstate significance and overestimate benefit

29. MPS/MSc in StatisticsAdaptive & Bayesian - Lect 329 References Whitehead (1997) Jennison and Turnbull (2000) See also Pocock and White (1999) who argue against ever using the Pocock test! For the analysis method, see Fairbanks and Madsen (1982) Tsiatis, Rosner and Mehta (1984)