Developing a progressive approach to using the GAIN in order to reduce the duration and cost of assessment with the GAIN short screener, Quick, and Computer Adaptive Testing Michael L. Dennis, Rodney R. Funk, Janet C. Titus, Barth B. Riley, Chestnut Health Systems, Bloomington, IL Sean Hosman, Sarah Kime, Assessments.com, Salt Lake City, UT Panel at the Joint Meeting on Adolescent Treatment Effectiveness, March 25-27, 2008, Washington, DC. This presentation supported by Center for Substance Abuse Treatment (CSAT), Substance Abuse and Mental Health Services Administration (SAMHSA) contracts and , as well as several individual CSAT, NIAAA, NIDA and private foundation grants. The opinions are those of the author and do not reflect official positions of the consortium or government. Available on line at or by contacting Joan Unsicker at 720 West Chestnut, Bloomington, IL 61701, phone: (309) , fax: (309) ,

Background  The Global Appraisal of Individual Needs (GAIN) is a family of instruments ranging from screeners to full standardized biopsychosocial.  Over the past decade it has become one of the most commonly used standardized instruments in adolescent substance abuse treatment and research.  Its strengths include - mapping onto common standards for diagnosis, treatment, planning, placement, and performance monitoring; - clear training, feedback and certification standards to support workforce development and interpretation; - a large number of investigators conducting primary studies, methodological and secondary analyses with it; - a large pooled data facilitating comparisons across studies, sites and evidenced based practices; and the introduction (in 2008) of a new web based infrastructure to support use across multiple settings.

Purpose of this panel  The biggest single complaint about the GAIN is the time it takes to administer ( minutes for a full GAIN) and the cost (both in terms of training and staff time to administer).  This provides an overview of the progressive approach to assessment we are trying to develop and then talk about three specific efforts we are working on to reduce the duration and cost of assessment. 1. Introduction of the GAIN Short Screener (GSS) 2. Developing a newer and more efficient version of the GAIN –Quick (GQ) 3. Developing a computer adaptive testing (CAT) version of the full GAIN

Progressive Assessment Approach  Screening to Identify Who Needs to Be Fully “Assessed” - Focus on brevity, simplicity for administration & scoring - Needs to be adequate for triage and referral  Quick Assessment for Targeted Referral - Assessment of who needs a feedback, brief intervention or referral for more specialized assessment or treatment - Needs to be adequate for brief intervention  Comprehensive Biopsychosocial - Used to identify common problems and how they are interrelated - Needs to be adequate for diagnosis, treatment planning and placement of common problems  Specialized Assessment - The bio-psycho-social may identify areas where additional assessment by a specialist (e.g., psychiatrist, school counselor) may be needed to rule out a diagnosis or develop a treatment plan or individual education plan  Program Level Assessment - For program management, evaluation, and planning

Reducing assessment time and cost by using the GAIN short screener Michael L. Dennis, Rodney R. Funk, Chestnut Health Systems, Bloomington, IL Sean Hosman, Sarah Kime, Assessments.com, Salt Lake City, UT

Overview of the GAIN-SS  A 3- to 5-minute screener  Used in general populations to identify or rule-out clients who will be identified as having a behavioral health disorders on the min versions of the GAIN  Easy for use by staff with minimal training or direct supervision  Provides a measure of change  Designed for self- or staff-administration, with paper and pen, computer, or on the web  Translated by collaborators into several languages including French, Japanese, Portuguese, and Spanish so far

Factor Structure of GAIN Measures of Psychopathology Source: Dennis, Chan, and Funk (2006)

Recency Response Set  Recency of 20 problems rated as past month (3), 2-12 months ago (2), more than a year ago (1), never (0)  Combined by cumulative time period as: - Past month count (3s) to measure of change - Past year count (2s or 3s) to predict diagnosis - Lifetime count (1s, 2s or 3s) as a measure of peak severity.  Can be classified within time period low (0), moderate (1-2) or high (3)  Can also be used to classify remission as - Early (lifetime but not past month) - Sustained (lifetime but not past year)

Item Selection Using a Rasch Measurement Model Items around key decision point

Internalizing Disorder Screening (IDScr) Externalizing Disorder Screening (EDScr)

Substance Disorder Screening (SDScr) Crime/violence Disorder Screening (CVScr)

Some important caveats…  Does not focus on low level problems (e.g., any use) to minimize false positives  Designed to have 90% sensitivity and is only self report, which means it does miss some and that clinicians should still be able to over ride on a case by case basis  Inclusion of lifetime measures provide checks against temporary phenomena (e.g., being in a controlled environment) and to drift when used to measure change (e.g., lifetime use should never go down)  Just a screener, not a full assessment

Example of GAIN SS Triage Profile

Example of Monitoring with GAIN SS Intake3 Mon Mon 15 Mon 18 Mon 21 Mon 24 Mon Total Disorder Screener (TDScr) 12+ mon.s ago (#1s) 2-12 Mon.s ago (#2s) Past Month (#3s) Lifetime (#1,2,or 3) Track Gap Between Prior and current Lifetime Problems to identify “under reporting” Track progress in reducing current (past month) symptoms) Monitor for Relapse

Using GAIN SS to Help with Placement Decisions for Adolescents Substance Users

Psychometric Properties Total Disorder Screener (TDScr) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Prevalence (% 1+ disorder) Sensitivity (% w disorder above) Specificity (% w/o disorder below) (n=6194 adolescents) Low Mod. High 99% prevalence, 91% sensitivity, & 89% specificity at 3 or more symptoms Using a higher cut point increases prevalence and specificity, but decreases sensitivity Total score has alpha of.85 and is correlated.94 with full GAIN version Source: Dennis et al 2006

GSS Performance by Subscale and Disorders Prevalence Sensitivity Specificity Screener/Disorder Internal Disorder Screener (0-5) Any Internal Disorder 81% 99% 94% 55% 71% 99% Major Depression 56% 87% 98% 72% 54% 94% Generalized Anxiety 32% 56% 100% 83% 44% 83% SuicideIdeation 24% 43% 100% 84% 41% 79% Mod/High Traumatic Stress 60% 82% 94% 60% 55% 90% External Disorder Screener (0-5) Any External Disorder 88% 97% 98% 67% 75% 96% AD, HD or Both 65% 82% 99% 78% 51% 85% Conduct Disorder 78% 91% 98% 70% 62% 90% Substance Use Disorder Screener (0-5) Any Substance Disorder 96% 100% 96% 68% 73% 100% Dependence 65% 87% 100% 91% 30% 82% Abuse 30% 13% 89% 25% 14% 28% Crime Violence Screener (0-5) Any Crime/Violence 88% 99% 94% 49% 76% 99% High Physical Conflict 31% 46% 100% 70% 38% 77% Mod/High General Crime 85% 100% 94% 51% 71% 100% Total Disorder Screener (0-5) Any Disorder 97% 99% 91% 47% 89% Any Internal Disorder 58% 63% 100% 98% 8% 28% Any External Disorder 68% 75% 100% 99% 10% 37% Any Substance Disorder 89% 92% 99% 92% 20% 51% Any Crime/Violence 68% 73% 100% 96% 10% 32% Low (0), Moderate (1-2), and High (3+) cut points can be used to identify the need for specific types of interventions Moderate can be targeted where resources allow or where a more assertive approach is desired Mod/Hi can be used to evaluate program delivery/referral

Rapidly Spreading  State or Provincial wide implementation in multiple states (ID, CT, LA, MI, MN, NH, NV, OR, SC, WA, WI) and provinces (BC, ON, QU) in one or more large systems (adolescent or adult addiction treatment, mental health, welfare, juvenile or criminal justice, Student or Employee Assistance Programs),  Used by SAP or EAP in Brazil, Canada, Japan, Mexico, United States and being translated for use in China.  Most have relied on paper and pencil administration or installed into their own information system – but then do not get reports.  There is a software template available, but its use still requires adaptation to one’s own system  Assessments.com is one of the one of the first commercial vendors to offer a web based version of the GAIN SS for both system and individual level user.

References  Dennis, M.L., Chan, Y-.F., & Funk, R.R. (2006). Development and validation of the GAIN Short Screener (GAIN-SS) for psychopathology and crime/violence among adolescents and adults. The American Journal on Addictions, 15(supplement 1), Available from N_Short_Screener.pdf. N_Short_Screener.pdf  Dennis, M. L., Feeney, T., Stevens, L. H., & Bedoya, L. (2007). Global Appraisal of Individual Needs– Short Screener (GAIN-SS): Administration and Scoring Manual for the GAINSS Version Bloomington, IL: Chestnut Health Systems. Retrieved on from Acknowledgements The above manual, articles and this presentation were supported by the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for Substance Abuse Treatment (CSAT) under contracts , , , and using data provided by the following grantees: CSAT (TI-11320, TI-11324, TI-11317, TI-11321, TI-11323, TI-11874, TI-11424, TI-11894, TI-11871, TI , TI-11423, TI-11432, TI-11422, TI-11892, TI-11888, TI013313, TI013309, TI013344, TI013354, TI013356, TI013305, TI013340, TI , TI03345, TI012208, TI013323, TI-14376, TI-14261, TI ,TI-14252, TI-14315, TI-14283, TI-14267, TI-14188, TI-14103, TI-14272, TI-14090, TI-14271, TI , TI-14196, TI-14214, TI-14254, TI-14311, TI-15678, TI-15670, TI-15486, TI-15511, TI-15433, TI , TI-15682, TI-15483,TI-15674, TI-15467, TI-15686, TI-15481, TI-15461, TI-15475, TI-15413, TI , TI-15514, TI-15672, TI-15478, TI-15447, TI-15545, TI-15671, TI-11320, TI-12541, TI00567); NIAAA (R01 AA 10368); NIDA (R37 DA11323, R01 DA ); the Illinois Criminal Justice Information Authority (95-DB-VX-0017); the Illinois Office of Alcoholism and Substance Abuse (PI 00567); the Intervention Foundation’s Drug Outcome Monitoring Study (DOMS); and the Robert Woods Johnson Foundation’s Reclaiming Futures project. Any opinions about these data are those of the authors and do not reflect official positions of the government or individual grantees. Thanks to Janet C. Titus, Joan I. Unsicker, Rod Funk, Ya-Fen Chan, Michelle White, Lexy Adkins, Tim Feeney and David Smith for their help in writing this manual. Thanks to Sandra McGuinness for developing the software application. Suggestions, comments, and questions can be sent to Dr. Michael Dennis, Chestnut Health Systems, 720 West Chestnut, Bloomington, IL 61701,