1. A Phase IV Meta Analytic Study of the Replicability of Motivational Enhancement Therapy/ Cognitive Behavior Therapy for 5 sessions (MET/CBT5) in 36 sites Michael L. Dennis, Ph.D., Melissa Ives, M.S.W. Chestnut Health Systems, Bloomington, IL and Randy Muck, M.Ed. Substance Abuse and Mental Health Services Administration (SAMHSA) Center for Substance Abuse Treatment (CSAT), Rockville, MD Joint Meeting on Adolescent Treatment Effectiveness, March 25, 2008, Washington, DC

2. Background In 1997 the third wave of cannabis use was the largest and youngest cohort to date, double the number of adolescents presenting to publicly funded treatment There were no publicly available manual guided evidenced based practices targeting this population The Cannabis Youth Treatment (CYT) experiments (n=600) were designed to manualize and field test five promising intervention for short term outpatient treatment of adolescent with cannabis (and other) substance use disorders Adapted from earlier studies with adult alcohol and cannabis users, Motivational Enhancement Therapy/ Cognitive Behavior Therapy for 5 sessions (MET/CBT5) was the briefest, one of the least expensive, similar in clinical outcomes, and hence one of the most cost-effective approaches evaluated (Dennis et al 2004; French et al 2003).

3. Effective Adolescent Treatment (EAT) From 2003 to 2008 SAMHSA’s Center for Substance Abuse Treatment (CSAT) conducted a phase IV (i.e., post randomization) replication of MET/CBT5 in 36 sites. All sites received standardized training, quality assurance and monitoring on their implementation of MET/CBT5, as well as the collection of data with the Global Appraisal of Individual Needs (GAIN) to facilitate comparison with the original CYT study in terms of implementation and outcome. The objectives of this presentation are to examine the extent to which the EAT sites were able to: 1.Replicate the implementation of MET/CBT5, 2.Address gaps identified in the original study related to continuing care, family involvement, and mental health services, 3.Replicate or improve on the outcomes.

4. Location of Sites AK AL AR AZ CA CO CT DC DE FL GA HI IA ID IL IN KS KY LA MA MD ME MI MN MO MS MT NC ND NE NH NJ NM NV NY OH OK OR PA RI SC SD TN TX UT VA VT WA WI WV WY CYT: 4 Sites EAT: 36 Sites

5. Sample Selection The Target Population Inclusion Criteria for including cases from the EAT data set were adolescents who: –Were assigned to MET/CBT in Outpatient and –Reported lifetime abuse or dependence symptoms and –Reported substance use in the last 90 days they were in the community and –Who were due for follow-up –4702 of 6150 (76%) meet all inclusion criteria, For logistical reasons, and additional 828 (17.6% of 4702) cases were excluded because they did not have a follow-up or were missing the dependent variable at follow-up. The final sample size for EAT was 3874 Adolescents with an average time to their last follow-up of 8.0 months. This group was compared using GAIN data to a cohort of 202 Adolescents from CYT that met the same criteria with an average time to their last follow-up of 11.4 months.

6. Demographics 0%10%20%30%40%50%60%70%80%90%100% Male* AA White Hisp Mixed Other 10-14 15-17 18-22 Race Groups* Age* CYT MET/CBT5 Outpatient (n=202) EAT MET/CBT5 Outpatient (n=3874) EAT more likely to be Hispanic (Mixed was not an option in CYT) * P<.05

7. Clinical Characteristics 0%10%20%30%40%50%60%70%80%90%100% Alcohol Marijuana Amphetamines Cocaine, Opioids, Other Neither Internalizing only Both Externalizing only None Unofficial Arrest/Police Contact Court/probation Correctional institution Primary Substance* Comorbidity Delinquency Level* CYT MET/CBT5 Outpatient (n=202) EAT MET/CBT5 Outpatient (n=3874) EAT Less Likely to be Presenting for Cannabis * P<.05

8. Comparing CYT and EAT In order to compare CYT and EAT, we want to look at both the central tendency (median) and the range. In the next several slides we have done this using Tukey Box Plots like the one shown here. Criteria MedianMiddle 50% “Range” In most cases we have scaled the response set relative to the average for MET/CBT5 in CYT -2.00 -1.50 -0.50 0.00 0.50 1.00 1.50 2.00 2.50 3.00

9. Records Data on Initiation, Engagement and Continuing Care -2.00 -1.50 -0.50 0.00 0.50 1.00 1.50 2.00 2.50 CYTEATCYTEATCYTEAT InitiationEngagementContinuing Care -2.00 -1.50 -0.50 0.00 0.50 1.00 1.50 2.00 2.50 (started within 2 weeks)(6+ weeks & 4+ sessions)(any Tx 91-180 days post intake)

10. Adolescent Reports on Treatment Received

11. Adolescent Reports on GAIN Treatment Indices -0.50 0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 CYTEATCYTEATCYTEAT Substance AbuseMental HealthPhysical Health -0.50 0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00

12. Comparison of Outcomes Evaluate the Increase in the Days of Abstinence from Intake to Last Observation. For all time periods days abstinent are adjusted by subtracting any days in a controlled environment during the period (average is less than 5 days). Change scores are calculated as last observation minus intake. The large sample sizes involved make even trivial differences statistically significant. Thus this comparison focuses as much on clinical significance by using effect sizes.

13. Within and Between Group Effect Size Calculations Effect sizes for within group change reported as: Cohen’s d = (M Last -M Intake )/ Std Dev. Intake where small=.2, moderate=.4 and large=.8 Effect sizes for group (G) differences in change scores (Last-Intake) are reported relative to the grand mean (GM) as: Cohen’s f =  (ABS(G (change) –GM (change) ))/ Std Dev. (GM Change) where small=.1, moderate=.2 and large=.4

14. CYT vs. EAT (f=.09) EAT Sample size is so large the 95% CI do not show here * P<.05 * * *

15. Range of Change Effect Sizes (d) by Site 0.00 0.20 0.40 0.60 0.80 1.00 1.20 1.40 4 CYT Sites (f=0.39) (median within site d=0.29) 36 EAT Sites (f=0.21) (median within site d=0.49) 0.00 0.20 0.40 0.60 0.80 1.00 1.20 1.40 Cohen’s d

16. Change in Days Abstinent by Amount of Continuing Care (f=.15)* 0 10 20 30 40 50 60 70 80 90 Intake*Last Follow-up Below CYT (<95% CI) (d=0.39) Similar to CYT (w/in 95% CI) (d=0.55) Above CYT (>95% CI) (d=0.53) * P<.05

17. Change in Days Abstinent by Amount of Family Services (f=.06) 0 10 20 30 40 50 60 70 80 90 Intake*Last Follow-up Below CYT (<95% CI) (d=0.44) Similar to CYT (w/in 95% CI) (d=0.53) Above CYT (> 95% CI) (d=0.52) * P<.05

18. Change in Days Abstinent by Amount of Mental Health Treatment (f=.05) 0 10 20 30 40 50 60 70 80 90 Intake*Last Follow-up* Below CYT (< 95% CI) (d=0.48) Similar to CYT (w/in 95% CI) (d=0.52) Above CYT (> 95% CI) (d=0.42) * P<.05

19. Change in Days Abstinent by Baseline Days of Abstinence (f=1.63)* 0 10 20 30 40 50 60 70 80 90 Intake*Last Follow-up* 0 days (d=N/A) 1-12 days (d=13.16) 13-44 days (d=3.45) 45+ days (d=0.11) * P<.05

20. Change in Days Abstinent by Baseline Days In Controlled Environment (f=.68)* 0 10 20 30 40 50 60 70 80 90 Intake*Last Follow-up* 0 days (d=0.41) 1-12 days (d=0.54) 13-44 days (d=0.96) 45+ days (d=2.98) * P<.05

21. Change in Days Abstinent by Gender (f=.11)* 0 10 20 30 40 50 60 70 80 90 IntakeLast Follow-up* Female (d=0.52) Male (d=0.44) * P<.05

22. Change in Days Abstinent by Race/Ethnicity (f=.22)* 0 10 20 30 40 50 60 70 80 90 Intake*Last Follow-up African American (d=0.45) White (d=0.42) Hispanic (d=0.64) Mixed (d=0.46) Other (d=0.23) Race/Ethnicity: * P<.05

23. Change in Days Abstinent by Age Group (f=.15)* 0 10 20 30 40 50 60 70 80 90 Intake*Last Follow-up* 10-15 (d=0.36) 15-17 (d=0.5) 18-22 (d=0.44) Age: * P<.05

24. Change in Days Abstinent by Most Severe Substance\ a (f=.35)* 0 10 20 30 40 50 60 70 80 90 Intake*Last Follow-up* Alcohol (d=0.2) Marijuana (d=0.54) Amphetamines (d=0.61) Cocaine, Opioids, Other (d=0.56) * P<.05 \a based on in descending order on most symptoms, frequency, recency, and preference

25. Change in Days Abstinent by Pattern of Psychiatric Comorbidity (f=.19)* 0 10 20 30 40 50 60 70 80 90 Intake*Last Follow-up* Neither (d=0.41) Internalizing Dis. Only (d=0.55) Both Internalizing & Externalizing Disorders (d=0.57) Externalizing Dis. Only (d=0.42) * P<.05

26. Adjusted Days Abstinent by Delinquency (f=.20)* 0 10 20 30 40 50 60 70 80 90 Intake*Last Follow-up* None (d=0.46) Unofficial delinquency (d=0.55) Arrest/Police contact (d=0.45) Court/Probation/Parole (d=0.41) Correctional Institution (d=0.70) * P<.05

27. Limitations Primarily relied on adolescent self report (plus some records on implementation). It would have been useful to have collateral or urine reports. First cut only examined days of abstinence, it is possible that increases in family treatment and mental health services impact other variables. Could have used other ways of adjusting for time in a controlled environment Some individual level variables are highly confounded with site (e.g., being Hispanic). Ideally we should (and will) combine site and individual level predictors in a multi-level model.

28. Conclusions On average, the EAT cohort did better than CYT on initiation and continuing care, then equal or slightly lower on most other measures. The big difference however was that the sites varied more in their implementation, with some actually providing much more than CYT and others much less. On average EAT sites had larger effect sizes than CYT, but in both cohorts there were large variations by site. The provision of additional continuing care was associated with larger effects; Increasing family services and mental health treatment was not associated with increased changes in the days abstinent. Several individual characteristics were associated with more change, in particular - few baseline days of abstinence, more days in a controlled environment, being Hispanic, having a drug problem, having internalizing disorders, and coming from a correctional institution.

29. This presentation was supported by the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for Substance Abuse Treatment (CSAT) under contracts 207-98-7047, 277-00-6500, 270-2003-00006, and 270-07-0191 using data provided by the following CSAT grantees: (CYT: TI-11320, TI-11317, TI-11321, TI-11323, TI-11324, EAT: TI-15413, TI-15415, TI-15421, TI-15433, TI-15438, TI-15446, TI-15447, TI-15458, TI-15461, TI-15466, TI-15467, TI-15469, TI-15475, TI-15478, TI-15479, TI-15481, TI-15483, TI-15485, TI-15486, TI-15489, TI-15511, TI-15514, TI-15524, TI-15527, TI-15545, TI-15562, TI-15577, TI-15586, TI-15670, TI-15671, TI-15672, TI-15674, TI-15677, TI-15678, TI-15682, TI-15686). Any opinions about these data are those of the authors and do not reflect official positions of the government or individual grantees. Thanks to Rod Funk, Mark Lipsey, Barth Riley, Michelle White and Ken Winters for their suggestions. Suggestions, comments, and questions can be sent to Dr. Michael Dennis, Chestnut Health Systems, 720 West Chestnut, Bloomington, IL 61701, mdennis@chestnut.org.mdennis@chestnut.org Acknowledgements