1. Moving the field from ‘no wrong door’ to the ‘best door’: An actuarial estimate of expected outcomes by level of care among adolescents presenting for substance abuse treatment Michael L. Dennis, Rodney R. Funk, and Laverne Hanes-Stevens, Chestnut Health Systems, Bloomington, IL Panel at the Joint Meeting on Adolescent Treatment Effectiveness, March 25-27, 2008, Washington, DC. This presentation supported by Center for Substance Abuse Treatment (CSAT), Substance Abuse and Mental Health Services Administration (SAMHSA) contracts 270- 2003-00006 and 270-07-0191, as well as several individual CSAT, NIAAA, NIDA and private foundation grants. The opinions are those of the author and do not reflect official positions of the consortium or government. Available on line at www.chestnut.org/LI/Posters or by contacting Joan Unsicker at 720 West Chestnut, Bloomington, IL 61701, phone: (309) 827-6026, fax: (309) 829-4661, e-Mail: junsicker@Chestnut.Org

2. Background  In practice, programs primarily refer people to the limited range of services they have readily available.  Knowing nothing about the person other than what door they walked through we can correctly predict 75% (kappa=.51) of the adolescent level of care placements.  The American Society for Addiction Medicine (ASAM) has tried to recommend placement rules for deciding what level of care an adolescent should receive based on expert opinion, but run into many problems including - difficulty synthesizing multiple pieces of information - inconsistencies between competing rules, - the lack of the full continuum of care to refer people to, - having to negotiate with the participant, families and funders over what they will do or pay for - there is virtually no actual data on the expected outcomes by level of care to inform decision making related to placement

3. Objectives  This presentation uses data from intake to 12 months collected with the Global Appraisal of Individual Needs (GAIN) with the ASAM statements and clusters just discussed by Hanes-Stevens and Funk in the preceding presentations.  The goal is to make an actuarial estimate of the expected outcomes for each individual for each potential level of care to inform clinical decision making related to placement.

4. Method  Started with the 8301 people in the Funk et al cluster analysis.  Dropped 2156 of the 8301 people in the cluster analysis who were not due or did not have at least one follow-up yet  Analysis done on 6,145 adolescents with 1 or more follow-ups (83% of those due) from 203 level of care x site combinations  Examined the actual level of care within each cluster, collapsing any that had less than 50 adolescents with follow-ups.  Used logistic regression on individual outcomes and linear regression to predict counts of positive outcomes based on actual level of care  Used coefficients from above analysis to compute predicted outcomes within each cluster based on each level of care within that cluster  Compared levels of care based on the predicted outcomes  Cohen’s f (.1=small,.2=moderate,.4= large)  Odds Ratio (0.8/1.2 – small, 0.5/2.0- large)

5. Simplified Levels of Care 0%20%40%60%80%100% A Low-Low B Low-Mod C Mod-Mod D Hi-Low E Hi-Mod F Hi-Hi (CC) G Hi-Mod (E/P) H Hi-Hi (I/P/M) Outpatient (OP)Intensive Outpatient (IOP) Outpatient Continuing Care (OPCC)Short Term Residential (STR) Long Term Residential (LTR) All higher levels STR & LTR IOP/OPCC

6. Outcomes  Washington Circle Group and National Commission on Quality Assurance (NCQA) from private insurance  National Outcome Monitoring System from States (NOMS), i ncluding SAMHSA Cost Bands  Government Performance and Results Act (GPRA)

7. Group 1. Cohen’s Effect Size f on Treatment received based on records In Treatment Outcome ABCDEFGH Initiation of Treatment (w/in 14 days) 0.190.230.130.220.130.160.030.16 Evidenced based treatment 0.370.750.670.640.390.490.480.76 Engagement in Tx (30+ days, 3+ sessions) 0.000.07 0.130.020.150.110.12 Continuing Care (90+ days later) 0.21 0.190.160.180.120.140.12 Count of Above (0-4) 0.080.220.300.280.170.320.200.33 Cohen’s f >.1 in bold

8. Group 2. Cohen’s Effect Size f on Treatment received based on self report In Treatment Outcome ABCDEFGH Early Treatment Satisfaction after 2 sessions (TxSI>55) 0.680.530.310.270.240.220.250.36 Treatment Satisfaction after 3 months (TxSS>10) 0.160.180.110.080.160.190.080.16 Abstinent or 50% Reduction in Sub. Freq. Scale at 3 months 0.020.200.150.240.230.120.300.13 Within Tx Cost Bands from SAMHSA/CSAT 0.470.400.470.290.250.300.190.30 Count of Above (0-4) 0.480.490.370.260.16 0.150.28 Cohen’s f >.1 in bold

9. Group 3. Cohen’s Effect Size f on Tx Outcomes In Treatment Outcome ABCDEFGH No AOD Use \1 0.030.060.160.210.140.050.150.14 No AOD related Prob. \1 0.000.110.090.130.120.110.100.11 No Health Problems \2 0.010.050.140.040.100.030.04 No Mental Health Prob. \2 0.150.190.180.120.160.190.100.07 No Illegal Activity \2 0.090.180.08 0.130.150.090.04 No JJ System Involve. \1 0.18 0.220.060.210.190.140.21 Living in Community \1 0.170.270.310.210.320.270.100.22 No Family Prob. \2 0.050.140.090.050.09 0.190.06 Vocationally Engaged \1 0.220.170.160.150.130.140.020.15 Social Support \2 0.100.060.070.110.050.080.050.06 Count of above 0.190.220.150.030.110.160.110.07 \1 Past month \2 Past 90 days Cohen’s f >.1 in bold

10. Group 3. Variance Explained in Tx Outcomes* \1 Past month \2 Past 90 days *All statistically Significant

11. Key Predictors of Outcomes: Baseline Characteristics (1 of 3) 0%20%40%60%80%100% Female African American (vs Mixed/Other) Caucasian (vs Mixed/Other) Hispanic (vs Mixed/Other) Age (per year) Alcohol Primary (vs. Cannabis) Other Drug Primary (vs. Cannabis) % of 18 Odds Ratio LT.5.5 to.8.8 to 1.21.2 to 2.0GT 2.0

12. Key Predictors of Outcomes: Baseline Characteristics (2 of 3) LT.5.5 to.8.8 to 1.21.2 to 2.0GT 2.0

13. Key Predictors of Outcomes: Baseline Characteristics (3 of 3) LT.5.5 to.8.8 to 1.21.2 to 2.0GT 2.0

14. Key Predictors of Outcomes: ASAM Tx Planning Cluster LT.5.5 to.8.8 to 1.21.2 to 2.0GT 2.0

15. Key Predictors of Outcomes: Level of Care within Tx Planning Cluster A to C LT.5.5 to.8.8 to 1.21.2 to 2.0GT 2.0

16. Key Predictors of Outcomes: Level of Care within Tx Planning Cluster D to F LT.5.5 to.8.8 to 1.21.2 to 2.0GT 2.0

17. Key Predictors of Outcomes: Level of Care within Tx Planning Cluster G to H 0%20%40%60%80%100% IOP/OPCC vs. OP in G Residential vs. OP in G IOP vs. OP in H OPCC vs. OP in H Residential vs. OP in H % of 18 Odds Ratio LT.5.5 to.8nsd1.2 to 2.0GT 2.0 LT.5.5 to.8.8 to 1.21.2 to 2.0GT 2.0

18. Key Predictors of Outcomes: Group 1 Treatment Received from Records 0%20%40%60%80%100% Initiation Evidence Based Treatment Engagement Continuing Care % of 14 Odds Ratio LT.5.5 to.8.8 to 1.21.2 to 2.0GT 2.0

19. Key Predictors of Outcomes: Group 2 Treatment Received from Self Report 0%20%40%60%80%100% Early Treatment Satisfaction TX Satisfaction at 3 months No/Reduce AOD at 3 months Within SAMSHA Tx Cost Bands % of 10 Odds Ratio LT.5.5 to.8nsd1.2 to 2.0GT 2.0 LT.5.5 to.8.8 to 1.21.2 to 2.0GT 2.0

20. Predicted Count of Positive Outcomes by Level of Care: Cluster A Low - Low (n=1,025)

21. Best Level of Care*: Cluster A Low - Low (n=1,025)

22. Predicted Count of Positive Outcomes by Level of Care: Cluster C Mod-Mod (n=1209) 2 3 4 5 6 7 8 9 10 OutpatientIntensive Outpatient Outpatient - Continuing Care Residential 2 3 4 5 6 7 8 9 10

23. Best Level of Care*: Cluster C Mod-Mod (n=1209)

24. Predicted Count of Positive Outcomes by Level of Care: Cluster F Hi-Hi (CC) (n=968) 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 OutpatientIntensive Outpatient Outpatient - Continuing Care Residential

25. Best Level of Care*: Cluster F Hi-Hi (CC) (n=968)

26. Predicted Count of Positive Outcomes by Level of Care: Cluster G. Hi-Mod (Env/PH) (n=749)

27. Best Level of Care*: Cluster G Hi-Mod (Env/PH) (n=749)

28. Planned use  Best fit will be used to recommend a level of care at the end of the GAIN Recommendation and Referral Summary  Table of 18 outcomes by level of care for the predicted cluster will be available to consider other options if a given recommendation is not available or there is a need to negotiate  If staff change the cluster type (may be relevant if there is new information or they are between two), the above can be recalculated

29. Limitations  Data limited to self report, thus it is important to inform (not control) clinical decision making  Not a representative sample  Not available yet for subtypes of a level of care (e.g., a specific evidenced based approach to treatment), young adults or adults  Ideally it needs to be tested prospectively

30. Conclusions  The relationship between multiple variables and outcomes is complex and not easily done by clinicians.  The 8 cluster groups based on ASAM treatment planning cells can help to predict outcome  It is feasible to make an actuarial estimate of treatment outcomes that has the potential to improve treatment outcomes  While there often is an advantage to one particular level of care placement, there is also a fair amount of overlap – suggesting the value of informed decisions (not a fixed rule).

31. The above presentation was supported by the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for Substance Abuse Treatment (CSAT) under contracts 207-98-7047, 277-00-6500, 270-2003-00006, and 270-07-0191 using data provided by the following grantees: CSAT TI-13190, TI-13305, TI-13308, TI- 13309, TI-13313, TI-13322, TI-13323, TI-13340, TI-13344, TI-13345, TI-13354, TI- 13356, TI-13601, TI-14090, TI-14103, TI-14188, TI-14189, TI-14196, TI-14214, TI- 14252, TI-14254, TI-14261, TI-14267, TI-14271, TI-14272, TI-14283, TI-14311, TI- 14315, TI-14355, TI-14376, TI-15348, TI-15413, TI-15415, TI-15421, TI-15433, TI- 15446, TI-15447, TI-15458, TI-15461, TI-15466, TI-15467, TI-15469, TI-15475, TI- 15478, TI-15479, TI-15481, TI-15483, TI-15485, TI-15486, TI-15489, TI-15511, TI- 15514, TI-15524, TI-15527, TI-15545, TI-15562, TI-15577, TI-15584, TI-15586, TI- 15670, TI-15671, TI-15672, TI-15674, TI-15677, TI-15678, TI-15682, TI-15686, TI- 16386, TI-16400, TI-16414, TI-16904, TI-16915, TI-16928, TI-16939, TI-16961, TI- 16984, TI-16992, TI-17046, TI-17055, TI-17070, TI-17071, TI-17334, TI-17433, TI- 17434, TI-17475, TI-17484). Any opinions about these data are those of the authors and do not reflect official positions of the government or individual grantees. Suggestions, comments, and questions can be sent to Dr. Michael Dennis, Chestnut Health Systems, 720 West Chestnut, Bloomington, IL 61701, mdennis@chestnut.org.mdennis@chestnut.org Acknowledgements