Abstract VARIABLE SELECTION FOR DECISION MAKING IN MENTAL HEALTH Lacey Gunter 1,2, Ji Zhu 1, and Susan Murphy 1,2 Departments of Statistics 1 and Institute for Social Research 2, University of Michigan, Ann Arbor Departments of Statistics 1 and Institute for Social Research 2, University of Michigan, Ann Arbor Introduction Algorithm For a complete variable selection method using this new ranking procedure, we suggest the following algorithm which we call New Method U : 1. Select important predictors of R in X using a predictive variable 1. Select important predictors of R in X using a predictive variable selection method selection method 2. Rank the variables in X using score U ; select the top k in rank 2. Rank the variables in X using score U ; select the top k in rank 3. Use a predictive variable selection method to select from important 3. Use a predictive variable selection method to select from important predictors chosen in step 1, A, and k interactions chosen in step 2 predictors chosen in step 1, A, and k interactions chosen in step 2 We compare this method versus a standardmethod: a Lasso of the main We compare this method versus a standard method: a Lasso of the main effects of X, A and the interactions between X and A New Methods We propose ranking the variables in X based on potential for a qualitative interaction with the treatment. We give a score for ranking the variables based on 2 factors for evaluating qualitative interactions 1. The magnitude of the interaction between the variable and treatment 2. The proportion of patients whose optimal treatment changes given knowledge of the variable These 2 factors are illustrated in the plots below. big interaction small interaction big interaction big proportion big proportion small proportion big proportion big proportion small proportion We estimate the interaction factor by: D j = change in the effect of the optimal treatment over range of variable X j D j = change in the effect of the optimal treatment over range of variable X j See plot below for illustration; a * is the overall optimal treatment, the blue and red lines represent the fitted model, green ticks represent observations We estimate the proportion factor by: P j = percentage of patients in the sample whose optimal treatment P j = percentage of patients in the sample whose optimal treatment changes when variable X j is added to the fitted model changes when variable X j is added to the fitted model For example, see plot below. We combine D j and P j to make a score, U j for each variable. The scores, U, can be used to rank the variables. This poster discusses variable selection for medical decision making; in particular, decisions regarding when to provide treatment and which treatment to provide patients with mental health disorders. Variable selection is often needed in this setting to reduce costs incurred by collecting unnecessary information and to inform clinicians which variables are important for individualizing treatment. We present a new technique designed to find variables that aid in decision making. We demonstrate the utility of this technique on data from a randomized controlled trial which compared three alternate treatments for chronic depression. We have 3 components: 1. observations X = (X 1, X 2,…, X p ), 2. treatment action, A, 3. response, R Policy : guidelines for choosing treatment, A, given observations, X observations, X Goal: find the policy which results in the highest average response average response Example: clinical trial to test two alternative drug treatments Goal: discover optimal treatment for any future patient Conclusion In this poster, we presented a new technique explicitly designed to select variables for decision making. We demonstrated this method on a depression data set. We found new method U did a better job eliminating interaction variables that are not important for prescribing treatment, which allow clinicians to focus on important variables that can help make treatment more individualized. Reasons for variable selection in decision making: ● limited resources ● better interpretability ● improved performance Predictive selection techniques have been proposed, but are only part of the puzzle. We need variables that help determine the optimal treatment for each patient, variables that qualitatively interact with the treatment. What is a qualitative interaction? Depression Study Results We demonstrate this method on data from a depression study to determine which variables might help decipher the optimal depression treatment for each patient. Aim of the Nefazodone CBASP trial (1) – to compare efficacy of three alternate treatments for chronic major depressive disorder (MDD): 1. Nefazodone, 1. Nefazodone, 2. Cognitive behavioral-analysis system of psychotherapy (CBASP) 2. Cognitive behavioral-analysis system of psychotherapy (CBASP) 3. Nefazodone + CBASP 3. Nefazodone + CBASP For our analysis we used data from 440 patient with: We used bootstrap sampling to minimize the variability of the results. On each of 100 bootstrap samples, we performed the following analysis: 1. run new method U and the standard method 2. record the interaction variables selected The plots below give the percentage of time each interaction was selected for each method. The green threshold lines in above plots were determined as follows: 1. Remove interaction effects from the data 1. Remove interaction effects from the data 2. Run methods on new data 2. Run methods on new data 3. Threshold: largest percentage of time a variable was selected 3. Threshold : largest percentage of time a variable was selected This helped assess the maximum selection percentage we expect to see when no interaction effects exist. Only interaction variables with selection percentages above these thresholds should be selected. Results: The standard method selected 30 interaction variables. The new method selected only 4 interaction variables: 2 indicators dealing with alcohol, a somatic anxiety score and an indicator dealing with specific phobia. For more details see (2). STATISTICSX baseline variables such as patient’s background, medical history, current symptoms, etc. A assigned treatment R patient's condition and symptoms post treatment References: (1) Keller, M.B., McCullough, J.P., Klein, D.N.et al.: A Comparison of Nefazodone, the (1) Keller, M.B., McCullough, J.P., Klein, D.N.et al.: A Comparison of Nefazodone, the Cognitive Behavioral-analysis System of Psychotherapy, and Their Combination Cognitive Behavioral-analysis System of Psychotherapy, and Their Combination for Treatment of Chronic Depression. N. Engl. J. Med. 342 (2000) for Treatment of Chronic Depression. N. Engl. J. Med. 342 (2000) (2) Gunter, L., Murphy, S.A., Zhu, J.: Variable Selection for Optimal Decision Making. (2) Gunter, L., Murphy, S.A., Zhu, J.: Variable Selection for Optimal Decision Making. Technical Report 463, University of Michigan Statistics Department Technical Report 463, University of Michigan Statistics Department maximum effect of treatment a* on R minimum effect of treatment a* on R D j = max effect – min effect min effect 2 out of 7 subjects would change choice of optimal treatment given X j Acknowledgements: We wish to thank Martin Keller and the investigators of [2] for use of their data, and gratefully acknowledge Bristol-Myers Squibb for helping fund the study. We also acknowledge financial support from NIH grants R21 DA019800, K02 DA15674, P50 DA10075, and NSF grant DMS and technical support from A. John Rush, MD, University of Texas Southwestern. Variable % Chosen StandardMethodMethodU 1 Gender Racial category Marital status 2,13 0,1 5 Body mass index 21 6 Age in years at screening 90 7 Treated current depression 20 8 Medication current depression Psychotherapy current depression Treated past depression Medication past depression Psychotherapy past depression Age of MDD onset Number of depressive episodes 16,22,140,0,0 17 Length current episode MDD type of current episode 19,181, MDD current severity 9,30, MDD chronic status 15,200,0 24 MDD threshold frequency Dysthymic disorder current Dysthymia initial onset Length current dysthymia episode Alcohol 28,4612,17 30 Drug Social phobia 11,282, Specific phobia 3,32 0,6 35 Obsessive compulsive Post traumatic stress 9,20, Generalized anxiety 28,150,0 40 Anxiety disorder NOS Panic disorder 26,270,0 43 Body dysmorphic current Anorexia or Bulimia nervosa Global assessment of function Main study diagnosis 5,80,0 48 Severity of illness Chronic or double depression Total HAMA score HAMA Sleep disturbance factor HAMA Psychic Anxiety Score HAMA Somatic Anxiety Score Total HAMD-24 score Total HAMD-17 score HAMD Cognitive Disturbance HAMD Retardation Score HAMD Anxiety/Somatic symptom IDSSR Total Score IDSSR Anxious depression type IDSSR General/Mood Cognition IDSSR Anxiety/Arousal Score IDSSR Sleep scores 10,30,0 X 64 baseline variables listed in the table to the right A Nefazodone vs. Nefazodone + CBASP R Last observed Hamilton’s Rating Scale for Depression score, post treatment X qualitatively interacts with the treatment if at least two subsets of X values result in different optimal treatments. No Non-qualitative Qualitative Interaction Interaction Interaction Interaction Interaction Interaction