1 Variable Selection for Tailoring Treatment S.A. Murphy, L. Gunter & J. Zhu May 29, 2008

2 Outline Motivation Need for Variable Selection Characteristics of a Tailoring Variable A New Technique for Finding Tailoring Variables Comparisons Discussion

3 Motivating Example

4 Simple Example Nefazodone - CBASP Trial Randomization Nefazodone Nefazodone + Cognitive Behavioral Analysis System of Psychotherapy (CBASP) 50+ baseline covariates, both categorical and continuous

5 Simple Example Nefazodone - CBASP Trial Which variables in X are important for choosing the optimal treatment? X patient’s medical history, severity of depression, current symptoms, etc. A Nefazodone OR Nefazodone + CBASP R depression symptoms post treatment

6 Need for Variable Selection In clinical trials many pretreatment variables are collected to improve understanding and inform future treatment Yet in clinical practice, only the most informative variables for tailoring treatment can be collected. A combination of theory, clinical experience and statistical variable selection methods can be used to determine which variables are important in tailoring.

7 Current Statistical Variable Selection Methods Current statistical variable selection methods focus only on finding good predictors of the response Also need variables to help determine which treatment is best for individual patients, e.g. tailoring variables Experts typically have knowledge on which variables are good predictors, but intuition about tailoring variables is often lacking

8 What is a Tailoring Variable? Tailoring variables help us determine which treatment is best Tailoring variables qualitatively interact with the treatment; different values of the tailoring variable result in different best treatments. No Interaction Non-qualitative Interaction Qualitative interaction

9 Qualitative Interactions We focus on two important factors –The magnitude of the interaction between the tailoring variable and the treatment indicator –The proportion of patients for whom the best choice of treatment changes given knowledge of the variable big interaction small interaction big interaction big proportion big proportion small proportion

10 Magnitude of the Interaction We estimate magnitude factor by: D j = change in the effect of the best treatment a*=1 over the range of variable X j maximum effect of treatment a* on R D j = max effect – min effect minimum effect of treatment a* on R

11 Proportion We estimate the proportion factor by: P j = percentage of patients in the sample whose best treatment changes when variable X j is considered Treatment A=0 is best for 2 out of 7 subjects even though treatment A=1 is best overall

12 Ranking Score U We combine D and P to make a score U for each X pretreatment variable. Variables are ranked by their score, U; higher U’s correspond to higher evidence of a qualitative interaction by the X variable. We use this ranking in a variable selection algorithm to select important tailoring variables.

13 Variable Selection Algorithm 1.Select important predictors of R from X using a predictive variable selection method (reducing noise in R) 2.Rank interactions between X and A using score U, select all with nonzero U. 3.Construct a combined ranking of variables selected in steps 1 and 2 4.Choose between variable subsets using a criterion that trades off number of variables and estimated maximal response due to tailoring.

14 Simulations Data simulated under wide variety of realistic decision making scenarios (with and without qualitative interactions) Compared: –Ranking method, U, using variable selection algorithm –Standard technique: Lasso on (X, A, X  A) 1000 simulated data sets: recorded percentage of time each variable’s interaction with treatment was selected for each method

15 Simulation Results × Binary Qualitative Interaction  Non-qualitative Interaction  Spurious Interaction × Continuous Qualitative Interaction  Non-qualitative Interaction  Spurious Interaction

16 Simulation Results Generative Model Ave(# of Spurious Interactions Selected) Standard Method New Method One Binary Qualitative Interaction Four Non-qualitative Interactions One Continuous Qualitative Interaction Four Non-qualitative Interactions

17 Nefazodone - CBASP Trial Aim of the Nefazodone CBASP trial – to compare efficacy of three alternate treatments for major depressive disorder (MDD): 1.Nefazodone, 2.Cognitive behavioral-analysis system of psychotherapy (CBASP) 3.Nefazodone + CBASP Which variables might help tailor the depression treatment to each patient?

18 Nefazodone - CBASP Trial For our analysis we used data from 440 patients with X 64 baseline variables A Nefazodone vs. Nefazodone + CBASP R Hamilton’s Rating Scale for Depression score, post treatment

19 Nefazodone - CBASP Trial Used bootstrap samples to produce a selection percentage for each variable. Permutated the rows of the X*A matrix to produce thresholds. The highest ranked spurious interaction is less than the 80% threshold in 80% of repeated permutations.

20 Nefazodone - CBASP Trial

21 Discussion This method provides a list of potential tailoring variables while reducing the number of false leads. Replication is required to confirm the usefulness of a tailoring variable. Our long term goal is to generalize this method so that it can be used with data from Sequential, Multiple Assignment, Randomized Trials as illustrated by STAR*D.

22 Susan Murphy at for more This seminar can be found at SPR0508.ppt Support: NIDA P50 DA10075, NIMH R01 MH and NSF DMS Thanks for technical and data support go to –A. John Rush, MD, Betty Jo Hay Chair in Mental Health at the University of Texas Southwestern Medical Center, Dallas –Martin Keller and the investigators who conducted the trial `A Comparison of Nefazodone, the Cognitive Behavioral-analysis System of Psychotherapy, and Their Combination for Treatment of Chronic Depression’

23 Interaction Plot

24 Interaction Plot