Experiments and Dynamic Treatment Regimes S.A. Murphy At NIAID, BRB December, 2007

2 Outline Dynamic Treatment Regimes Challenges in Experimentation Defining Effects and Aliasing Example

3 Dynamic treatment regimes are individually tailored treatments, with treatment type and dosage changing with ongoing subject need. Mimic Clinical Practice. High variability across patients in response to any one treatment Relapse is likely without either continuous or intermittent treatment for a large proportion of people. What works now may not work later Exacerbations in disorder may occur if there are no alterations in treatment

4 The Big Questions What is the best sequencing of treatments? What is the best timings of alterations in treatments? What information do we use to make these decisions?

5 Two stages of treatment for each individual Observation available at j th stage Treatment (vector) at j th stage Primary outcome Y is a specified summary of decisions and observations

6 A dynamic treatment regime is a vector of decision rules, one per decision where each decision rule inputs the available information and outputs a recommended treatment decision.

7 Challenges in Experimentation

8 Dynamic Treatment Regimes (review) Constructing decision rules is a multi-stage decision problem in which the system dynamics are unknown. High quality data is likely provided by sequential multiple assignment randomized trials: randomize at each decision point— à la full factorial.

9 ExTENd Ongoing study at U. Pennsylvania (D. Oslin) Goal is to learn how best to help alcohol dependent individuals reduce alcohol consumption.

10 Oslin ExTENd Late Trigger for Nonresponse 8 wks Response TDM + Naltrexone CBI Random assignment: CBI +Naltrexone Nonresponse Early Trigger for Nonresponse Random assignment: Naltrexone 8 wks Response Random assignment: CBI +Naltrexone CBI TDM + Naltrexone Naltrexone Nonresponse

11 Challenges in Experimentation Dynamic Treatment Regimes are multi-component treatments: many possible components decision options for improving patients are often different from decision options for non-improving patients (T 2 is a vector and differs by outcomes observed during initial treatment) multiple components employed simultaneously medications, behavioral therapy, adjunctive treatments, delivery mechanisms, motivational therapy, staff training, monitoring schedule……. Future: series of screening/refining, randomized trials prior to confirmatory trial --- à la Fisher/Box

12 Screening experiments (review) 1)Goal is to eliminate inactive factors (e.g. components) and inactive effects. 2)Each factor at 2 levels 3)Screen main effects and some interactions 4)Design experiment using working assumptions concerning the negligibility of certain higher order factorial effects. 5)Designs and analyses permit one to determine aliasing (caused by false working assumptions) 6)Minimize formal assumptions

13 Stage 1 Factors: T 1 ={ A, B, C, D}, each with 2 levels Stage 1 outcome: Stage 2 Factors: T 2 = {F 2 --only if R=1, G 2 —only if R=0}, each with 2 levels Primary Outcome: Y continuous (2 6 = 64 simple dynamic treatment regimes) Simple Example for Two Stages

14 Two Stage Design: I=ABCDF 2 =ABCDG 2 A B C D F 2 =G

15 Screening experiments Can we: design screening experiments using working assumptions concerning higher order effects & determine the aliasing and provide an analysis method?

16 Defining the Effects

17 Reality

18 Defining the stage 2 effects Simple case: two stages of treatment with only one factor at each stage, a early measure of response, R at the end of stage 1 and a primary outcome Y. The potential outcomes are Define effects involving T 2 in a saturated linear model for

19 Defining the stage 2 effects Suppose the factors T 1 and T 2 are randomized. Assume consistency (Robins, 1997) then

20 Defining the stage 2 effects Define (factorial) effects involving T 2 in a saturated linear model

21 Defining the stage 2 effects

22 Defining the stage 1 effects (T 1 )

23 Defining the stage 1 effects

24 Defining the stage 1 effects Define effects involving only T 1 in a saturated linear model The above is equal to when {T 1, T 2 } are randomized and T 2 has a discrete uniform distribution on {-1,1}.

25 Defining the stage 1 effects In general when {T 1, T 2 } are randomized

26 Why marginal, why uniform? Define effects involving only T 1 via 1)The defined effects are causal. 2)The defined effects are consistent with tradition in experimental design for screening. –The main effect for one treatment factor is defined by marginalizing over the remaining treatment factors using a discrete uniform distribution.

27 Surprisingly both stage 1 and 2 effects can be represented in one (nonstandard) linear model: Representing the effects

28 where Causal effects: Nuisance parameters: and

29 Two Stage Design: I=ABCDF 2 =ABCDG 2 A B C D F 2 =G

30 General Formula Saturated model (for both R=0, 1) Z 1 matrix of stage 1 factor columns, Z 2 is the matrix of stage 2 factor columns, Y is a vector, p is the vector of response rates Classical saturated factorial effects model

31 Aliasing

32 Aliasing (Identification of Effects) In classical designs, the defining words immediately yield the aliasing. For example I=ABCDF means that only the sum of the main effect of F and the four way interaction, ABCD can be identified. The two effects can not be separately estimated. We can do the same here with a caveat.

33 Aliasing (Identification of Effects) The rows of {Z 1, Z 2 } are determined by the experimental design; each column in Z 1 is associated with a stage 1 factorial effect and similarly each column in Z 2 is associated with a stage 2 factorial effect. Fractional factorial designs lead to common columns in {Z 1, Z 2 }; these columns are given by the defining words.

34 Aliasing (Identification of Effects) Consider designs for which the defining words indicate a shared column in both Z 1 and Z 2 only if the column in Z 1 can be safely assumed to have a zero η coefficient or if the column in Z 2 can be safely assumed to have a zero β, α coefficient. The defining words then provide the aliasing in this case.

35 Six Factors: Stage 1: T 1 ={ A, B, C, D}, each with 2 levels Stage 2: T 2 = {F 2 --only for stage 1 responders, G 2 --only for stage 1 nonresponders}, each with 2 levels (2 6 = 64 simple dynamic treatment regimes) Simple Example for Two Stages

36 Two Stage Design: I=ABCDF 2 =ABCDG 2 A B C D F 2 =G

37 Simple Example This design has defining words: 1=ABCDF 2 =ABCDG 2 The labels of the identical columns in Z 1, Z 2 are CD=ABG 2, ABC=DG 2, F 2 DC=AB………..

38 Formal Assumptions for this Design We use this design if we can assume that all three way and higher order stage 2 effects are negligible (ABG 2, ABF 2, ABCG 2, ABCF 2, ….)—these are all β and α parameters. assume all four way and higher order effects involving R and stage 1 factors are negligible (R- p)ABCD, (R-p)ABC, (R-p)ABD …--- these are all η parameters. 1=ABCDF 2 =ABCDG 2

39 Aliasing for this Design Stage 1 main effects and stage 2 four-way interactions are aliased (e.g. A=BCDF 2, etc.). Stage 1 two way interactions and stage 2 three way interactions are aliased (e.g. CD=ABG 2, AB=CDF 2, etc.) Stage 1 three way interactions and stage 2 two way interactions are aliased (e.g. ABC=DF 2, ABC=DG 2, etc.) The Stage 1 four way interaction and stage 2 main effects are aliased (e.g. ABCD=F 2, ABCD=G 2 ). 1=ABCDF 2 =ABCDG 2

40 Working Assumptions for this Design Stage 2 three-way and four-way interactions are negligible. Stage 1 three-way and four-way interactions are negligible. If the working assumptions are correct then we will be able to estimate the “non-aliased” main and two-way interactions for both stage 1 and stage 2.

41 Analysis for this design Recall that Many columns in Z 1, Z 2 are identical (hence the aliasing of effects). Eliminate all multiple copies of columns and label remaining columns as stage 1 (or stage 2) main and two-way interaction effects. Replace response rates in p by observed response rates. Fit model.

42 Interesting Result in Simulations In simulations formal assumptions are violated. Response rates (probability of R=1) across 16 cells range from.55 to.73 Results are surprisingly robust to violations of formal assumptions. The maximal value of the correlation between 32 estimators of effects was.12 and average absolute correlation value is.03 Why? Binary R variables can not vary that much. If response rate is constant, then the effect estimators are uncorrelated as in classical experimental design.

43 Discussion Compare this to using observational studies to construct dynamic treatment regimes –Uncontrolled selection bias (causal misattributions) –Uncontrolled aliasing. Secondary analyses would assess if other variables collected during treatment should enter decision rules.

44 Joint work with –Derek Bingham (Simon Fraser) And informed by discussions with –Vijay Nair (U. Michigan) –Bibhas Chakraborty (U. Michigan) –Vic Strecher (U. Michigan) This seminar can be found at: lsa.umich.edu/~samurphy/seminars/NIAID12.07.ppt