1. An Experimental Paradigm for Developing Dynamic Treatment Regimes S.A. Murphy Univ. of Michigan March, 2004

2. Dynamic Treatment Regimes

3. Setting: Management of chronic, relapsing disorders such as alcohol, cocaine addiction and mental illness Characteristics: May need a sequence of treatments prior to improvement Improvement marred by relapse Intervals during which more intense treatment is required alternate with intervals in which less treatment is sufficient

4. Dynamic Treatment Regimes are individually tailored treatments, with treatment type and dosage changing with ongoing subject need. Mimic Clinical Practice. Brooner et al. (2002) Treatment of Opioid Addiction Breslin et al. (1999) Treatment of Alcohol Addiction Prokaska et al. (2001) Treatment of Tobacco Addiction Rush et al. (2003) Treatment of Depression

5. EXAMPLE : Treatment of alcohol dependency. Primary outcome is a summary of heavy drinking scores over time

7. GOAL : Provide experimental methods for developing treatment assignment, i.e. decision, rules.

8. k Decisions Observations made prior to j th decision Action at j th decision Primary Outcome: for a known function u

9. A dynamic treatment regime is a vector of decision rules, one per decision If the strategy is implemented then for j=1,…., k.

11. Challenges

12. Two Challenges Delayed Effects ---sequential within-person randomization Dynamic Treatment Regimes are High Dimensional Multi-component Treatments ---series of randomized developmental trials prior to confirmatory trial.

13. Delayed Effects Why choosing the best initial treatment on the basis of a randomized trial of initial treatments and choosing the best secondary treatment on the basis of a randomized trial of secondary treatments will not provide the best dynamic treatment regime. Why you need data in which all treatment sequences are possible.

16. Summary: Evaluating the initial treatments requires that we first calculate the mean primary outcome of patients for each combination of secondary treatment, intermediate outcome and initial treatment. The main point : If the mean primary outcome to secondary treatment vary by initial treatment then we should use sequential within-person randomization.

17. When might the mean primary outcome to each combination of secondary treatment, intermediate outcome vary by initial treatment?

20. Delayed Effects: The Bottom Line Are there unobserved but potentially important common causes of the primary and intermediate outcomes? Are there unobserved but potentially important causal pathways from initial treatment to primary outcome? If yes to either of the above then use sequentially within-person randomized trials to develop good dynamic treatment regimes.

21. An Aside!

24. Adaptive Treatment Strategies are High Dimensional Multi-Component Treatments when to start treatment? which treatment to start? when to step-up treatment? which step-up treatment? when to step down treatment to maintenance/monitoring? which maintenance/monitoring treatment? what information to use to make each of the above decisions?

25. Meeting the Challenges Delayed Effects: Sequential within-person randomization: Randomize at each decision point. High Dimensionality: Series of developmental randomized trials prior to a confirmatory trial (Box, Hunter and Hunter,1978, pg. 303).

26. Examples of sequentially within-person randomized trials: CATIE (2001) Treatment of Psychosis in Alzheimer’s Patients CATIE (2001) Treatment of Psychosis in Schizophrenia STAR*D (2001) Treatment of Depression Thall et al. (2000) Treatment of Prostate Cancer

27. Test Statistic and Sample Size Formula for Primary Analysis

28. Proposal Primary analysis in developmental trial is to discriminate between regimes with different initial treatments. In primary analysis consider dynamic treatment regimes with decision rules depending only on summaries of X j ’s (say S j ’s) In secondary analyses consider dynamic treatment regimes with decision rules depending on the X j ’s. Choose randomization probabilities to equalize the sample size across possible strategies.

29. Proposal Estimate the mean of Y when the decision rules, are followed: The variance of the estimator is used to construct a sample size formula.

30. Randomization probability of treatment A j =a j given past: Estimating Function:

31. Solve, We obtain:

32. Primary Analysis: Test statistic to compare two strategies with different initial treatments: See Murphy, van der Laan and Robins (2001) for technical details.

33. Calculating Sample Sizes:

35. In this example: Balanced design hence choosing the randomization probabilities to equalize the sample size across all possible regimes yields uniform randomization probabilities. An estimator of the mean of Y under the decision rules is the average response of individuals whose treatment pattern is consistent with the rules:

36. The average response for individuals whose treatment pattern is consistent with the rules: is the number of treatment alternatives at decision j is response variance under treatment regime

37. Sample Size Formula: whereis the Type I error and is the power of the test to detect a difference in the mean response between regimes. In our simple example:

38. Secondary Hypotheses Compare dynamic treatment regimes that begin with the same treatment; in this example, compare primary outcome to secondary treatments by levels of the summary intermediate outcome. Use an analysis that tests if other intermediate outcomes differentiate for whom each secondary treatment is best and if any pretreatment information differentiates for whom each initial treatment is best. (Murphy, 2003; Robins, 2003)

39. Discussion Simulations indicate that sample size formula is accurate for balanced designs. Secondary analyses can only explore dynamic treatment regimes that comply with the restrictions imposed by the experimental design. Trial design and analyses targeted at scientific goal.

40. Open Problems This setting requires development/generalization of Box's experimentation approach of several developmental trials, all based on randomization prior to a confirmatory trial. How could one use working assumptions on the form of delayed effects to speed up the developmental process? Use working assumptions to pool information. What kinds of working assumptions make sense? How do you detect potential violations of the working assumptions?

41. Open Problems Choosing randomization probabilities in unbalanced designs. Dealing with high dimensional X-- feature extraction- -in secondary analyses.

42. This seminar can be found at http://www.stat.lsa.umich.edu/~samurphy/seminars/seminar0 304.ppt The paper can be found at http://www.stat.lsa.umich.edu/~samurphy/papers/Experiment alEvidence.pdf

43. Principles in Designing a Sequentially Within-Person Randomized Trial

44. Secondary treatment alternatives should vary by only a simple low dimension summary (responder status) instead of all intermediate outcomes (adherence, burden, craving, etc.). Collect intermediate outcomes that might be useful in ascertaining for whom each treatment works best; information that might enter into the decision rules.

45. Principles in Designing a Sequentially Within-Person Randomized Trial Choose a primary hypothesis that is both scientifically interesting and aids in developing the adaptive treatment strategy. Choose secondary hypotheses that further develop the adaptive treatment strategy and use the randomization to reduce confounding.