1. Experiments and Dynamic Treatment Regimes S.A. Murphy Univ. of Michigan Florida: January, 2006

2. 2 Joint work with –Derek Bingham (Simon Fraser) –Linda Collins (PennState) And informed by discussions with –Vijay Nair (U. Michigan) –Bibhas Chakraborty (U. Michigan) –Vic Strecher (U. Michigan)

3. 3 Outline Dynamic Treatment Regimes Challenges in Experimentation Defining Effects and Aliasing Examples

4. 4 Dynamic treatment regimes are individually tailored treatments, with treatment type and dosage changing with ongoing subject need. Mimic Clinical Practice. Dynamic Treatment Regimes

5. 5 Two stages of treatment for each individual Observation available at j th stage Treatment (vector) at j th stage Primary outcome Y is a specified summary of decisions and observations

6. 6 A dynamic treatment regime is a vector of decision rules, one per decision where each decision rule inputs the available information and outputs a recommended treatment decision.

7. 7 Challenges in Experimentation

8. 8 Dynamic Treatment Regimes (review) Constructing decision rules is a multi-stage decision problem in which the system dynamics are unknown. Analysis methods for observational data dominate statistical literature (Murphy, Robins, Moodie & Richardson, Tsiatis) Better data provided by sequential multiple assignment randomized trials: randomize at each decision point— à la full factorial.

9. 9 Reality

10. 10 Challenges in Experimentation Dynamic Treatment Regimes are multi-component treatments: many possible components decision options for improving patients are often different from decision options for non-improving patients (T 2 differs by outcomes observed during initial treatment) multiple components employed simultaneously medications, adjunctive treatments, delivery mechanisms, behavioral contingencies, staff training, monitoring schedule……. Future: series of screening/refining, randomized trials prior to confirmatory trial --- à la Fisher/Box

11. 11 Screening experiments (review) 1)Goal is to eliminate inactive factors (e.g. components) and inactive effects. 2)Each factor at 2 levels 3)Screen marginal causal effects 4)Design experiment using working assumptions concerning the negligibility of certain effects. (Think ANOVA) 5)Designs and analyses permit one to determine aliasing (caused by false working assumptions) 6)Minimize formal assumptions

12. 12 Six Factors: M 1, E, C, T, A 2 (only for R=1), M 2 (only for R=0), each with 2 levels (2 6 = 64 simple dynamic treatment regimes) The budget permits 16 cells --16 simple dynamic treatment regimes. Simple Example

13. 13 Design: I=M 2 M 1 ECT=A 2 M 1 ECT M 1 E C T A 2 =M 2 - - - - + - - - + - - - + - - - - + + + - + - - - - + - + + - + + - + - + + + - + - - - - + - - + + + - + - + + - + + - + + - - + + + - + - + + + - - + + + + +

14. 14 Screening experiments Can we: design screening experiments using working assumptions concerning the marginal causal effects & provide an analysis method that permits the determination of the aliasing??

15. 15 Defining the Effects

16. 16 Defining the stage 2 effects Two decisions (two stages): (R=1 if quick response to T 1 ) Define effects involving T 2 in an ANOVA decomposition of

17. 17 Defining the stage 2 effects Define effects involving T 2 in an ANOVA decomposition of E[Y|T 1,R,T 2 ] = R(β 1 T 2 + β 2 T 2 T 1 ) + (1-R) (α 1 T 2 + α 2 T 2 T 1 ) +.....

18. 18 Defining the stage 1 effects (T 1 )

19. 19 Defining the stage 1 effects

20. 20 Defining the stage 1 effects Define Define effects involving only T 1 in an ANOVA decomposition of

21. 21 Defining the stage 1 effects Intuition: If T 2 were randomized with probability ½ among responders (R=1) and T 2 were randomized with probability ½ among nonresponders (R=0) then (“ignore” R and future treatment).

22. 22 Why marginal, why uniform? Define effects involving only T 1 in an ANOVA decomposition of 1)The defined effects are causal. 2)The defined effects are consistent with tradition in experimental design for screening. –The main effect for one treatment factor is defined by marginalizing over the remaining treatment factors using a discrete uniform distribution.

23. 23 Why marginal, why uniform? 2)The defined effects are marginal consistent with tradition in experimental design for screening. –The main effect for one treatment factor is defined by marginalizing over the remaining factors using an uniform distribution. When there is no R, the main effect for treatment T 1 is

24. 24 An Aside: Ideally you’d like to replace by (X 2 is a vector of intermediate outcomes) in defining the effects of T 1.

25. 25 Use an ANOVA-like decomposition: Representing the effects

26. 26 where Causal effects: Nuisance parameters: and

27. 27 General Formula New ANOVA Z 1 matrix of stage 1 treatment columns, Z 2 is the matrix of stage 2 treatment columns, Y is a vector Classical ANOVA

28. 28 Aliasing {Z 1, Z 2 } is determined by the experimental design The defining words (associated with a fractional factorial experimental design) identify common columns in the collection {Z 1, Z 2 } ANOVA

29. 29 Aliasing ANOVA Consider designs with a shared column in both Z 1 and Z 2 only if the column in Z 1 can be safely assumed to have a zero η coefficient or if the column in Z 2 can be safely assumed to have a zero β, α coefficient. The defining words provide the aliasing in this case.

30. 30 Simple Examples

31. 31 Six Factors: M 1, E, C, T, A 2 (only for R=1), M 2 (only for R=0), each with 2 levels (2 6 = 64 simple dynamic treatment regimes) The budget permits 16 cells --16 simple dynamic treatment regimes. Simple Example

32. 32 Assumptions A 2 C, A 2 T, M 2 E, M 2 T and CE along with the main effects in stage 1 and 2 are of primary interest. Working Assumption: All remaining causal effects are likely negligible. Formal Assumption: Consider designs for which a shared column in Z 1 and Z 2 occurs only if the column in Z 1 can be safely assumed to have a zero η coefficient (concerns interactions of stage 1 factors with R) or if the column in Z 2 can be safely assumed to have a zero β/α coefficient (stage 2 effects).

33. 33 Design 1 No formal assumptions. I=M 1 ECT The design column for A 2 =M 2 is crossed with stage 1 design. A 2 T is aliased with A 2 M 1 EC. The interaction A 2 T is of primary interest and the working assumption was that A 2 M 1 EC is negligible. CE is aliased with M 1 T. The interaction CE is of primary interest and the working assumption was that M 1 T is negligible.

34. 34 Design 2 Formal assumption: No three way and higher order stage 2 causal effects & no four way and higher order effects involving R and stage 1 factors. I=M 2 M 1 ECT=A 2 M 1 ECT A 2 T are aliased with M 1 CE; the interaction A 2 T is of primary interest and the working assumption was that M 1 CE is negligible. M 2 M 1 T is negligible so CE is not aliased.

35. 35 Interesting Result in Simulations In simulations formal assumption are violated. Response rates (probability of R=1) across 16 cells range from.55 to.73 Results are surprisingly robust to a violation of formal assumptions. The maximal value of the correlation between 32 estimators of effects was.12 and average absolute value is.03 Why? Binary response variables can not vary that much. If response rate is constant, then this design and analysis method reduces to standard experimental design and analysis!

36. 36 Discussion In classical screening experiments we Screen marginal causal effects Design experiment using working assumptions concerning the negligibility of the effects. Designs and analyses permit one to determine aliasing Minimize formal assumptions We can do this as well when screening for multi-stage decision problems.

37. 37 Discussion Compare this to using observational studies to construct dynamic treatment regimes –Uncontrolled selection bias (causal misattributions) –Uncontrolled aliasing. Secondary analyses would assess if variables collected during treatment should enter decision rules. This seminar can be found at: http:// www.stat. lsa.umich.edu/~samurphy/seminars/FloridaState01.06.ppt

38. 38 Reality