1. Abstract VARIABLE SELECTION FOR DECISION MAKING IN MENTAL HEALTH Lacey Gunter 1,2, Ji Zhu 1, and Susan Murphy 1,2 Departments of Statistics 1 and Institute for Social Research 2, University of Michigan, Ann Arbor Departments of Statistics 1 and Institute for Social Research 2, University of Michigan, Ann Arbor Introduction Algorithm For a complete variable selection method using this new ranking procedure, we suggest the following algorithm which we call New Method U : 1. Select important predictors of R in X using a predictive variable 1. Select important predictors of R in X using a predictive variable selection method selection method 2. Rank the variables in X using score U ; select the top k in rank 2. Rank the variables in X using score U ; select the top k in rank 3. Use a predictive variable selection method to select from important 3. Use a predictive variable selection method to select from important predictors chosen in step 1, A, and k interactions chosen in step 2 predictors chosen in step 1, A, and k interactions chosen in step 2 We compare this method versus a standardmethod: a Lasso of the main We compare this method versus a standard method: a Lasso of the main effects of X, A and the interactions between X and A New Methods We propose ranking the variables in X based on potential for a qualitative interaction with the treatment. We give a score for ranking the variables based on 2 factors for evaluating qualitative interactions 1. The magnitude of the interaction between the variable and treatment 2. The proportion of patients whose optimal treatment changes given knowledge of the variable These 2 factors are illustrated in the plots below. big interaction small interaction big interaction big proportion big proportion small proportion big proportion big proportion small proportion We estimate the interaction factor by: D j = change in the effect of the optimal treatment over range of variable X j D j = change in the effect of the optimal treatment over range of variable X j See plot below for illustration; a * is the overall optimal treatment, the blue and red lines represent the fitted model, green ticks represent observations We estimate the proportion factor by: P j = percentage of patients in the sample whose optimal treatment P j = percentage of patients in the sample whose optimal treatment changes when variable X j is added to the fitted model changes when variable X j is added to the fitted model For example, see plot below. We combine D j and P j to make a score, U j for each variable. The scores, U, can be used to rank the variables. This poster discusses variable selection for medical decision making; in particular, decisions regarding when to provide treatment and which treatment to provide patients with mental health disorders. Variable selection is often needed in this setting to reduce costs incurred by collecting unnecessary information and to inform clinicians which variables are important for individualizing treatment. We present a new technique designed to find variables that aid in decision making. We demonstrate the utility of this technique on data from a randomized controlled trial which compared three alternate treatments for chronic depression. We have 3 components: 1. observations X = (X 1, X 2,…, X p ), 2. treatment action, A, 3. response, R Policy : guidelines for choosing treatment, A, given observations, X observations, X Goal: find the policy which results in the highest average response average response Example: clinical trial to test two alternative drug treatments Goal: discover optimal treatment for any future patient Conclusion In this poster, we presented a new technique explicitly designed to select variables for decision making. We demonstrated this method on a depression data set. We found new method U did a better job eliminating interaction variables that are not important for prescribing treatment, which allow clinicians to focus on important variables that can help make treatment more individualized. Reasons for variable selection in decision making: ● limited resources ● better interpretability ● improved performance Predictive selection techniques have been proposed, but are only part of the puzzle. We need variables that help determine the optimal treatment for each patient, variables that qualitatively interact with the treatment. What is a qualitative interaction? Depression Study Results We demonstrate this method on data from a depression study to determine which variables might help decipher the optimal depression treatment for each patient. Aim of the Nefazodone CBASP trial (1) – to compare efficacy of three alternate treatments for chronic major depressive disorder (MDD): 1. Nefazodone, 1. Nefazodone, 2. Cognitive behavioral-analysis system of psychotherapy (CBASP) 2. Cognitive behavioral-analysis system of psychotherapy (CBASP) 3. Nefazodone + CBASP 3. Nefazodone + CBASP For our analysis we used data from 440 patient with: We used bootstrap sampling to minimize the variability of the results. On each of 100 bootstrap samples, we performed the following analysis: 1. run new method U and the standard method 2. record the interaction variables selected The plots below give the percentage of time each interaction was selected for each method. The green threshold lines in above plots were determined as follows: 1. Remove interaction effects from the data 1. Remove interaction effects from the data 2. Run methods on new data 2. Run methods on new data 3. Threshold: largest percentage of time a variable was selected 3. Threshold : largest percentage of time a variable was selected This helped assess the maximum selection percentage we expect to see when no interaction effects exist. Only interaction variables with selection percentages above these thresholds should be selected. Results: The standard method selected 30 interaction variables. The new method selected only 4 interaction variables: 2 indicators dealing with alcohol, a somatic anxiety score and an indicator dealing with specific phobia. For more details see (2). STATISTICSX baseline variables such as patient’s background, medical history, current symptoms, etc. A assigned treatment R patient's condition and symptoms post treatment References: (1) Keller, M.B., McCullough, J.P., Klein, D.N.et al.: A Comparison of Nefazodone, the (1) Keller, M.B., McCullough, J.P., Klein, D.N.et al.: A Comparison of Nefazodone, the Cognitive Behavioral-analysis System of Psychotherapy, and Their Combination Cognitive Behavioral-analysis System of Psychotherapy, and Their Combination for Treatment of Chronic Depression. N. Engl. J. Med. 342 (2000) 331-366 for Treatment of Chronic Depression. N. Engl. J. Med. 342 (2000) 331-366 (2) Gunter, L., Murphy, S.A., Zhu, J.: Variable Selection for Optimal Decision Making. (2) Gunter, L., Murphy, S.A., Zhu, J.: Variable Selection for Optimal Decision Making. Technical Report 463, University of Michigan Statistics Department Technical Report 463, University of Michigan Statistics Department maximum effect of treatment a* on R minimum effect of treatment a* on R D j = max effect – min effect min effect 2 out of 7 subjects would change choice of optimal treatment given X j Acknowledgements: We wish to thank Martin Keller and the investigators of [2] for use of their data, and gratefully acknowledge Bristol-Myers Squibb for helping fund the study. We also acknowledge financial support from NIH grants R21 DA019800, K02 DA15674, P50 DA10075, and NSF grant DMS 0505432 and technical support from A. John Rush, MD, University of Texas Southwestern. Variable % Chosen StandardMethodMethodU 1 Gender 210 2 Racial category 111 3-4 Marital status 2,13 0,1 5 Body mass index 21 6 Age in years at screening 90 7 Treated current depression 20 8 Medication current depression 230 9 Psychotherapy current depression 160 10 Treated past depression 161 11 Medication past depression 100 12 Psychotherapy past depression 380 13 Age of MDD onset 50 14-16 Number of depressive episodes 16,22,140,0,0 17 Length current episode 142 18-19 MDD type of current episode 19,181,1 20-21 MDD current severity 9,30,1 22-23 MDD chronic status 15,200,0 24 MDD threshold frequency 80 25 Dysthymic disorder current 40 26 Dysthymia initial onset 230 27 Length current dysthymia episode 10 28-29 Alcohol 28,4612,17 30 Drug 10 31-32 Social phobia 11,282,3 33-34 Specific phobia 3,32 0,6 35 Obsessive compulsive 510 36-37 Post traumatic stress 9,20,0 38-39 Generalized anxiety 28,150,0 40 Anxiety disorder NOS 260 41-42 Panic disorder 26,270,0 43 Body dysmorphic current 80 44 Anorexia or Bulimia nervosa 223 45 Global assessment of function 50 46-47 Main study diagnosis 5,80,0 48 Severity of illness 10 49 Chronic or double depression 60 50 Total HAMA score 32 51 HAMA Sleep disturbance factor 30 52 HAMA Psychic Anxiety Score 31 53 HAMA Somatic Anxiety Score 3414 54 Total HAMD-24 score 41 55 Total HAMD-17 score 40 56 HAMD Cognitive Disturbance 100 57 HAMD Retardation Score 20 58 HAMD Anxiety/Somatic symptom 10 59 IDSSR Total Score 20 60 IDSSR Anxious depression type 50 61 IDSSR General/Mood Cognition 54 62 IDSSR Anxiety/Arousal Score 03 63-64 IDSSR Sleep scores 10,30,0 X 64 baseline variables listed in the table to the right A Nefazodone vs. Nefazodone + CBASP R Last observed Hamilton’s Rating Scale for Depression score, post treatment X qualitatively interacts with the treatment if at least two subsets of X values result in different optimal treatments. No Non-qualitative Qualitative Interaction Interaction Interaction Interaction Interaction Interaction