1. Using a cluster analysis based case-mix solution to facilitate the evaluation and development of adolescent substance abuse treatment programs. Michael L. Dennis, Ph.D. Chestnut Health Systems, Bloomington, IL

2. Objectives 1.Identification of Clients with similar presenting pathology based on a cluster analysis of the GAIN’s core psychiatric and behavior scales. 2.Demonstration of how the “case-mix” of these subgroups impacts program averages. 3.Illustration of how psychiatric case mix groups can be used to aid program evaluation and planning within or across program evaluation.

3. Global Appraisal of Individual Needs (GAIN) A standardized bio-psycho-social that integrates clinical and research assessment for diagnosis, placement, treatment planning, process measures, outcome monitoring, and economic evaluation. Core sections include cognitive assessment, background/access, substance use, physical health, risk behaviors, mental health, environment, legal, vocational, staff ratings Over 100 scales/indices, with alpha over.9 on main scales and over.7 on subscales Test retest data suggest reliability of items/scales over.7 Self reported use consistent with urine, salvia, and collateral reports (Kappa of.81 or more) Predicts blind diagnosis of co-occurring psychiatric disorders including ADHD (kappa = 1.00), Mood Disorders (kappa = 0.85), Conduct Disorder or Oppositional Defiant Disorder (kappa = 0.82), Adjustment Disorder (kappa = 0.69), and No other diagnosis (kappa = 0.91)

4. Factor Structure and Cluster Analysis based on 2968 Clients from 61 Treatment Units Adolescent Inpatient/Therapeutic Community Adolescent Outpatient/IOP Adult Outpatient/IOP/OP Methadone Treatment Adult Inpatient/Therapeutic Community Oakland, CA Shiprock, NMLos Angeles, CA Phoenix/Tempe, AZ Tucson, AZ Miami, FL St. Petersburg, FL Cantonsville, MD Baltimore, MD New York, NY Chicago, IL Peoria, IL Maryville, IL Philadelphia, PA Bloomington, IL Farmington, CT

5. Hypothesized Structure of the GAIN’s Psychopathology Measures * Main scales have alpha over.85, subscales over.7 Behavioral Complexity Crime and Violence Internal Mental Distress

6. Confirmatory Factor Analysis (CFA) Comparative Fit Index:.974 Root Mean Square Error of Approximation: 0.079.60 Internal.27 HSTI.67 DSI.77 ASI.47 TSI.51 External.68 CDI.83 IAI.60 HII.25 Crime/Violence.55 DCI.62 ICI.62 PCI.39 GCTI.55 SA Problems.78 SDIY.51 SAIY.64 SIIY.54 SSI.54 General Severity.50 ri re rv rs.71.78.74.68.88.52.82.73.88.71.62.91.46.23.80.74.63.79 Comparative Fit Index:.97 vs.98 Parsimony Ratio:.80 vs.70 CFI x PR:.78 vs.68 Root Mean Square Error of Approximation:.04 vs.04 Invariant vs Variant Across Age and Level of Care

7. Creating Cluster Code Types The overall severity and four core dimensions were used to create 7 code types with Ward’s minimum distance cluster analysis. Total and four dimensional scores triaged into low, medium and high based on +/-.5 standard deviations from the mean Code types labeled most common group as: –High, medium or low overall severity on total score –Labeled in order from highest to lowest severity dimension –Lines // used to separate those in high/ medium/ low severity on each each of four dimensions –Sample size Discriminate Function Analysis for Classifying New Cases (Kappa =.82)

8. 7 Cluster Code Types High G., CV, BC ID, SP// (N=214) 8% High F. ID, BC, SP, CV// (N=336) 12% High E. CV, BC, SP/ ID/ (N=429) 15% Med. D. SP/ BC, ID/ CV (N=471) 17% Low A. //CV, ID, BC, SP (N=545) 19% Low B. SP/ID/ CV, BC (N=370) 13% Med. C. /BC, CV/ID, SP (N=467) 16% High to Low Severity order Hi / Med / Low range divided by // Code Type (A,B,C..)

9. General Severity by Code Type

10. Substance Problem (SP) by Code Type

11. Internal Distress (ID) by Code Type

12. Behavior Complexity (BC) by Code Type

13. Behavior Complexity (CV) by Code Type

14. Case Mix by Age and Level of Care

15. Sponsored By: Center for Substance Abuse Treatment (CSAT), Substance Abuse and Mental Health Services Administration (SAMHSA), U.S. Department of Health and Human Services (DHHS) Adolescent Treatment Model Program Sites ATM 1999 1998 Miami, FL Bloomington, IL Cantonsville, MD Tempe, AZ Shiprock, NM Baltimore, MD Los Angeles, CA Oakland, CA New York, NY Tucson, AZ

16. ATM involved the full range of Code Types

17. Evaluating Cluster Code Types Severity should go up with level of care (LOC) – one of the most commonly used case mix variables. The cluster code type should do better than LOC in terms of: –Maximizing individual differences between cluster subgroups –Minimizing individual indifference by LOC within cluster subgroups The cluster code types should help to predict differential response patterns to treatment

18. Case Mix Severity Goes up With Level of Care 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Early InterventionOP/IOPLTRSTR G-High CV,BC,ID,SP// F-High ID,BC,SP/CV/-- E-High CV,BC,SP/ID/ D-Mod SP/BC,ID/CV C-Mod BC/CV,ID/SP B-Low /SP,ID/CV,BC A-Low //CV,ID,BC,SP PCM Index Score PCM Index Score (Weighted Average)

19. Level of Care Is Related to “Average” Severity -4.0 -3.0 -2.0 0.0 1.0 2.0 3.0 4.0 Total Score (f=0.4) SP. Substance Problem (f=0.26) ID. Internal Distress (f=0.29) BC Behavior Complexity (f=0.28) CV. Crime/Violence (f=0.14) Z-score OP (n=553) LTR (n=373) STR (n=573) Individual Differences explained by LOC quantified with Cohen’s effect size f

20. However Cluster Subgroups are More Distinct From Each Other -4.0 -3.0 -2.0 0.0 1.0 2.0 3.0 4.0 Total Score (f=1.75) SP. Substance Problem (f=0.48) ID. Internal Distress (f=1.19) BC Behavior Complexity (f=1.85) CV.Crime Violence (f=1.19) Z-score A-Low //CV,ID,BC,SP (n=208) B-Low /SP,ID/CV,BC (n=101) C-Mod BC/CV,ID/SP (n=286) D-Mod SP/BC,ID/CV (n=252) E-High CV,BC,SP/ID/ (n=281) F-High ID,BC,SP/CV/-- (n=180) G-High CV,BC,ID,SP// (n=191) +338%+85% +310% +561% +750% Cohen’s effect size f increased by 85% to 750%

21. -4.0 -3.0 -2.0 0.0 1.0 2.0 3.0 4.0 Total Score (f=0.05) SP. Substance Problem (f=0.04) ID. Internal Distress (f=0.11) BC Behavior Complexity (f=0.16) CV. Crime/Violence (f=0.04) Z-score OP (n=114) LTR (n=59) STR (n=35) A-Low //CV,ID,BC,SP Once we account for subgroup, LOC differences are gone and Cohen’s effect size f goes down

22. B-Low /SP,ID/CV,BC -4.0 -3.0 -2.0 0.0 1.0 2.0 3.0 4.0 Total Score (f=0.08) SP. Substance Problem (f=0.12) ID. Internal Distress (f=0.06) BC Behavior Complexity (f=0.02) CV. Crime/Violence (f=0.09) Z-score OP (n=38) LTR (n=23) STR (n=40)

23. C-Mod BC/CV,ID/SP -4.0 -3.0 -2.0 0.0 1.0 2.0 3.0 4.0 Total Score (f=0.18) SP. Substance Problem (f=0.13) ID. Internal Distress (f=0.22) BC Behavior Complexity (f=0.13) CV. Crime/Violence (f=0.09) Z-score OP (n=138) LTR (n=82) STR (n=66)

24. D-Mod SP/BC,ID/CV -4.0 -3.0 -2.0 0.0 1.0 2.0 3.0 4.0 Total Score (f=0.17) SP. Substance Problem (f=0.18) ID. Internal Distress (f=0.14) BC Behavior Complexity (f=0.1) CV. Crime/Violence (f=0.1) Z-score OP (n=78) LTR (n=57) STR (n=117)

25. E-High CV,BC,SP/ID/ -4.0 -3.0 -2.0 0.0 1.0 2.0 3.0 4.0 Total Score (f=0.13) SP. Substance Problem (f=0.22) ID. Internal Distress (f=0.14) BC Behavior Complexity (f=0.08) CV. Crime/Violence (f=0.08) Z-score OP (n=103) LTR (n=50) STR (n=128)

26. F-High ID,BC,SP/CV/ -4.0 -3.0 -2.0 0.0 1.0 2.0 3.0 4.0 Total Score (f=0.06) SP. Substance Problem (f=0.18) ID. Internal Distress (f=0.05) BC Behavior Complexity (f=0.06) CV. Crime/Violence (f=0.08) Z-score OP (n=43) LTR (n=44) STR (n=93)

27. G-High CV,BC,ID,SP// -4.0 -3.0 -2.0 0.0 1.0 2.0 3.0 4.0 Total Score (f=0.15) SP. Substance Problem (f=0.28) ID. Internal Distress (f=0.1) BC Behavior Complexity (f=0.13) CV. Crime/Violence (f=0.06) Z-score OP (n=39) LTR (n=58) STR (n=94)

28. Cluster Subgroups Significantly Reduces the Individual Differences Associated with Level of Care -100.0% -80.0% -60.0% -40.0% -20.0% 0.0% 20.0% 40.0% 60.0% 80.0% 100.0% Total Score SP. Substance Problem ID. Internal Distress BC Behavior Complexity CV. Crime/Violence Change in LOC Effect Size f A-Low //CV,ID,BC,SP (n=208) B-Low /SP,ID/CV,BC (n=101) C-Mod BC/CV,ID/SP (n=286) D-Mod SP/BC,ID/CV (n=252) E-High CV,BC,SP/ID/ (n=281) F-High ID,BC,SP/CV/-- (n=180) G-High CV,BC,ID,SP// (n=191)

29. Outpatient by Cluster Types Differentiates initial severity, and differences in response

30. Long Term Residential by Cluster Types Can identify subgroups (E, B) that are a higher risk of relapse or having other problems

31. Short Term Residential by Cluster Types Different levels of care/programs may do well (A,F,G) or have problems (B,C,D, E) with different subgroups

32. For a Given Subtype, it can identify when a particular level of care (or program) appears to do better. C-Mod BC/CV,ID/SP by LOC However this is still quasi- experimental and the adjustments are often imperfect

33. Conclusions Clustering people based on presenting problems appears to work better than level of care for describing initial case mix but is also correlated with it. Clinical subtype clusters can help to identify subgroups for which a program works well and/or where continuing care or other services may be needed. Within a clinical subtype, comparisons of level of care (programs, services etc) could be used to guide placement decisions and/or identify promising areas for experimentation.

34. Contact Information Michael L. Dennis, Ph.D. Lighthouse Institute, Chestnut Health Systems 720 West Chestnut, Bloomington, IL 61701 Phone: (309) 827-6026, Fax: (309) 829-4661 E-Mail: mdennis@chestnut.org A copy of these slides will be posted at: www.chestnut.org/li/posters

35. Errata The following additional slide was presented by the discussant, Dr. Mark Fishman, to show how case mix varied at the program level even within level of care.

36. Case Mix by Level of Care/ATM program Early Intervention at the low end STR/LTR dominates high end Also demonstrates that Level of Care is only a rough proxy of case mix