1. A Phase 4 replication of MET/CBT5 in 36 sites to examine how findings vary by site, client characteristics, and implementation fidelity Michael L. Dennis, Ph.D., Melissa Ives, M.S.W. Chestnut Health Systems, Normal, IL Richard D. Lennox, Ph.D. Psychometric Technologies, Hillsborough, NC Randy Muck, M.Ed. Substance Abuse and Mental Health Services Administration (SAMHSA) Center for Substance Abuse Treatment (CSAT), Rockville, MD College of Problems on Drug Dependence (CPDD) & Society of Adolescent Substance Abuse Treatment Effectiveness (SASATE), Reno, NV. June 23, 2009

2. Background In 1997 the third wave of cannabis use was the largest and youngest cohort to date, double the number of adolescents presenting to publicly funded treatment There were no publicly available manual guided evidenced based practices targeting this population The Cannabis Youth Treatment (CYT) experiments (n=600) were designed to manualize and field test five promising intervention for short term outpatient treatment of adolescent with cannabis (and other) substance use disorders Adapted from earlier studies with adult alcohol and cannabis users, Motivational Enhancement Therapy/ Cognitive Behavior Therapy for 5 sessions (MET/CBT5) was the briefest, one of the least expensive, similar in clinical outcomes, and hence one of the most cost-effective approaches evaluated (Dennis et al 2004; French et al 2003).

3. Effective Adolescent Treatment (EAT) From 2003 to 2008 SAMHSA’s Center for Substance Abuse Treatment (CSAT) conducted a phase IV (i.e., post randomization) replication of MET/CBT5 in 36 sites. All sites received standardized training, quality assurance and monitoring on their implementation of MET/CBT5, as well as the collection of data with the Global Appraisal of Individual Needs (GAIN) to facilitate comparison with the original CYT study in terms of implementation and outcome. The objectives of this presentation are to : 1.Demonstrate that EAT used MET/CBT5 with a more diverse population 2.Replicate the implementation and outcomes of MET/CBT5 3.Identify participant characteristics moderators and intervention mediators that are associated with outcomes

4. Sample Selection The Target Population Inclusion Criteria for including cases from the EAT data set were adolescents who: –Were assigned to MET/CBT in Outpatient and –Reported lifetime abuse or dependence symptoms and –Reported substance use in the last 90 days they were in the community and –Who were due for 6 month follow-up Of 36 sites, 12 were dropped because they did not collect treatment received data at 3 months or because they less than 50% with BOTH 3 and 6 month interview Of the remaining 3556 clients from 24 sites, –2540 (71%) have outcome data at 6 months –2540 (86%) have outcome data at 3 or 6 months This group was compared using GAIN data to a cohort of 199 (98% of 202) adolescents from CYT that met the same criteria

5. EAT More Geographically Diverse AK AL AR AZ CA CO CT DC DE FL GA HI IA ID IL IN KS KY LA MA MD ME MI MN MO MS MT NC ND NE NH NJ NM NV NY OH OK OR PA RI SC SD TN TX UT VA VT WA WI WV WY CYT: 4 Sites Included EAT: 24 Sites Excluded EAT: 12 Sites

6. Demographics Race groups* Age groups* *p<.01 EAT Clients were more likely to be female, non-white, and have a wider age range

7. Clinical Characteristics Primary Substance Comobidity Delinquency Level* *p<.01 EAT Clients less likely to have cannabis as primary substance, similar in their comorbidity, and to have more justice system involvement.

8. EAT did as well or Better as CYT on Service Engagement *p<.01

9. Days of Treatment in the First 3 Months *p<.01 84% 94%

10. Content of CYT’s 5 Treatments Varied *p<.01 Little related to family services Low end for external or wrap around services Similar on direct services

11. MET/CBT5 in CYT and EAT had a Similar Range of Content *p<.01 EAT received less external services

12. Comparison of Outcomes Evaluate the Increase in the Days of Abstinence from Intake to Last Observation. For all time periods days abstinent are adjusted by subtracting any days in a controlled environment during the period (average is less than 5 days). Change scores are calculated as last observation minus intake. The large sample sizes involved make even trivial differences statistically significant. Thus this comparison focuses as much on clinical significance by using effect sizes.

13. Within and Between Group Effect Size Calculations Effect sizes for within group change reported as: Cohen’s d = (M Last -M Intake )/ Std Dev. Intake where small=.2, moderate=.4 and large=.8 Effect sizes for group (G) differences in change scores (Last-Intake) are reported relative to the grand mean (GM) as: Cohen’s f =  (ABS(G (change) –GM (change) ))/ Std Dev. (GM Change) where small=.1, moderate=.2 and large=.4

14. Change in Days Abstinent by Study (f=.02) Slopes are NOT significantly different EAT more severe

15. Effect Size by Site

16. Intake Predictors of Change in Days Abstinent Between Intake and Last Follow-up VariableCohen’s fβP Days Abstinent0.48-0.630.0000 Controlled Environment0.21-0.090.0000 Count of Clinical Problems0.32-0.080.0000 Cocaine Problem Severity0.27-0.630.0000 Any Opioid Use0.16-0.040.0248 Other Drug Severity\a0.10-0.050.0016 Age0.05-0.050.0035 Gender0.04-0.060.0006 R 2.33--0.0000 \a Other than alcohol or marijuana

17. Change in Days Abstinent by Days of Abstinence at Intake (f=.48)* * P<.0001

18. Change in Days Abstinent by Days in a Controlled Environment at Intake (f=. 21)* * P<.0001

19. Change in Days Abstinent by Count of Major Problems at Intake (f=.32)* * P<.0001

20. Change in Days Abstinent by Cocaine/Crack Problem Severity at Intake (f=.27)* * P<.0001

21. Change in Days Abstinent by Any Opioid Use in community at Intake (f=.16)* * P<.0001

22. Change in Days Abstinent by Other Drug Problem Severity at Intake (excluding alcohol/marijuana) (f=.10)* * P<.0001

23. Change in Days Abstinent by Age group at intake (f=.05)

24. Change in Days Abstinent by Gender (f=.04)* * P<.05

25. Other Client Characteristics that did NOT Predict the Amount of Change Race Single Parent Metropolitan size Primary drug Days of use or problem group for alcohol, cannabis, amphetamine Victimization Psychopathology Delinquency levels

26. Baseline + Mediators of Change in Days Abstinent Between Intake and Last Follow-up VariableCohen’s fβP Baseline Risk\a--.65.0000 Days in a Controlled Environment Mons 1-3 0.22-.17.0000 High Treatment Cost0.04-.07.0008 Any incarceration0.17-.06.0075 Any other Treatment0.04.03.0431 Site (6+Other)0.14--.0000 R 2.42--.0000 \a Predicted from intake only

27. Change in Days Abstinent by Days in a Controlled Environment Mons 1-3 (f=.22)* * P<.0001

28. Change in Days Abstinent by Costs within SAMHSA cost bands (f=.04) * P<.001

29. Change in Days Abstinent by Any Incarceration in Mons 1-3 (f=.17)* * P<.001

30. Change in Days Abstinent by Any Other SA Treatment (f=.04)* * P<.05

31. Change in Days Abstinent by Sites (f=.14)* * P<.0001 6 sites had more change than average 2 sites had less change than average (all low severity)

32. Other Moderators that did NOT Predict the Amount of Change Initiation & Engagement in but p>.05 Length of stay and continuing care Treatment Received Scales (direct, family, wrap around) SA Days of residential, IOP, OP, Medication, ER or Urine test/breathalyzer or summary index MH Days of Inpatient, OP, Medication, ER or summary index PH Days of Inpatient, OP, Medication, ER or summary index Study or other sites Months from intake to last follow-up

33. Limitations Primarily relied on adolescent self report (plus some records on implementation). It would have been useful to have collateral or urine test results. First cut only examined days of abstinence, it is likely that different variables impact other outcomes Could have used other ways of adjusting for time in a controlled environment All variation by site not explained yet. May need to look at environment and peer risk to explain differences

34. Conclusions EAT grantees were more diverse geographically, demographically and clinically EAT grantees implementation was better than CYT in terms of engagement, similar in dosage, and only slightly less in content Baseline severity was the primary factor explaining differences in the amount of change observed Engagement, dosage and content were not the major mediator of change – environmental variables were Further investigation is needed to understand why some sites did better than average even after controlling for above factors

35. This presentation was supported by the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for Substance Abuse Treatment (CSAT) under contracts 207-98-7047, 277-00-6500, 270-2003-00006, and 270-07-0191 using data provided by the following CSAT grantees: (CYT: TI-11320, TI-11317, TI-11321, TI-11323, TI-11324, EAT: TI-15413, TI-15415, TI-15438, TI-15446, TI-15447, TI-15458, TI-15461, TI-15466, TI-15469, TI-15478, TI-15479, TI-15481, TI-15483, TI-15489, TI-15514, TI-15524, TI-15545, TI-15562, TI-15577, TI-15586, TI- 15672, TI-15677, TI-15682, TI-15686). Any opinions about these data are those of the authors and do not reflect official positions of the government or individual grantees. Thanks to Rod Funk, Mark Lipsey, Barth Riley, Michelle White and Ken Winters for their suggestions. Suggestions, comments, and questions can be sent to Dr. Michael Dennis, Chestnut Health Systems, 448 Wylie Dr., Normal, IL 61761, mdennis@chestnut.org.mdennis@chestnut.org Acknowledgements