Global Appraisal of Individual Needs (GAIN) Michael L. Dennis, Ph.D. Chestnut Health Systems, Bloomington, IL

The Global Appraisal of Individual Needs or GAIN is actually a series of standardized instruments designed to integrate the assessment for both clinical (e.g., diagnosis, bio-psycho-social assessment, placement, and treatment planning) and program evaluation (needs assessment, clustering, fidelity, outcomes, and benefit cost) purposes.

Objectives 1.Provide an overview of the GAINs key features and organization. 2.Highlight some key methodological findings from current adolescent treatment work using the GAIN 3.Briefly demonstrate Capabilities of Computer Applications

Key Features and Organization.

Development and Purpose of the GAIN The GAIN family of instruments were developed through a 10 year collaboration of researchers, clinicians, policy makers, and IT specialists They provide a standardized approach to measuring: –Eligibility/need (i.e., screening), –DSM/ICD Diagnosis, –ASAM level of care Placement, –Study/State/Federal Reporting, –Treatment Planning, –Severity/Case Mix, –Change in Functioning, Service Utilization, and other Outcomes, and –Economic Cost and Benefits of treatment.

Methodological Features It can be used and has norms available across age groups and level of care, It has 103 scales with demonstrated reliability and validity and over 3 dozen scientist doing further research on it, It is designed to be modularized so you can use all or parts of it and transfer data (e.g, from screener to full assessment), It has a clear training and certification program, has technical assistance/support, and It is available at minimal cost.

Administration/Logistical Features Administration can be done by paper/pencil, by computer, on a stand alone PC, network, and the web (via other contractors), HIPPA compliant data base Data can be transferred to/from multiple MIS systems or other providers, Computerized scoring, narrative interpretative reports, intervention specific reports, validity and re- keying reports are available, Has versions (varying in content) that can take from 20 to 120 minutes, and It is design for administration by a paraprofessional but so that a range of behavioral, health and other professionals can use/ interpret it with minimal additional questions.

The Progression of Substance Use Problems Multiple Problem Clients Clinical Disorder Problem Use Frequent Use Bingeing Opportunistic Use Experimentation No Use Severity

Progressive Assessment Screening to Identify Who Needs Fully Assessed –Focus on brevity, simplicity for administration Screening for Targeted Referral –Assessment of who needs crisis or brief intervention (e.g., by SAP, doctor) vs. more detailed assessment and specialized treatment/referral –Decision rules about where to send may be more complex (e.g., substance abuse, mental health, both) Comprehensive Biopsychosocial –Used to identify common problems and how they are inter-related –Requires more skill in administration and even more in interpretation Specialized Assessment –The bio-psycho-social may identify areas where additional assessment by a specialist (e.g., psychiatrist, school counselor) may be needed to rule out a diagnosis or develop a treatment plan or individual education plan Program Level Assessment –For program management, evaluation and planning

Organization of the core GAIN A.Administration (including records information, cognitive impairment, calendaring, referral information, general instructions) B. Background and Treatment Arrangements (demographics, custody, access to care) S.Substance Use (including treatment readiness, relapse potential, withdrawal, abuse, and dependence, treatment history, content and satisfaction with recent treatment, current medication) P.Physical Health (including disabilities, current and childhood infectious diseases, allergies, lifetime history, treatment history, current medication) R. Risk Behaviors and Disease Prevention (including needle and sexual risk behaviors, sexual preference, birth control, tobacco use/dependence, fasting and exercise, testing and prevention classes)

Organization- Continued M. Mental Health and Emotions (including somatic, depressive, suicide risk, anxiety, traumatic distress, ADHD, CD, personality disorder, treatment history, current medication) E.Environment and Living Situation (including housing, homelessness, public/emergency housing, use in home, controlled environment, children status, living, vocational, and social risk, violence towards others, traumatic victimization, other psycho-social stressors, general social support, spirituality, general satisfaction) L. Legal (Civil & Criminal) (civil court involvement, illegal activities, status offenses, arrest history, current criminal justice involvement, outstanding warrants and payments) V.Vocational (School, Work, Financial) (educational attainment/degrees, school problems and involvement, military history, vocational attainment, work problems and involvement, current vocational status, financial problems, pathological gambling, TANF participation, personal and family income, HHS poverty index, drug/alcohol expenses) Z. End (administrative time, comments, signatures, administrative ratings and methods information, diagnostic impressions, special study information)

Within Section Organization Status –Recency (past prevalence) –Breadth (symptom count/covariate) –Current prevalence (days or times) –ASAM or diagnostic check boxes for hand scoring Utilization –Lifetime History –Recency –Current utilization Cross Item Ratings (substance problems, satisfaction) Treatment Planning (urgency, wants) Staff Ratings (urgency, denial and misrepresentation)

Alternative Versions GAIN-M90 for outcome monitoring interviews GAIN-CI for collateral initial interview GAIN-CM for Collateral outcome monitoring interviews GAIN-Quick for screening, outreach and other areas where a briefer (10-20 minute) assessment is desired GAIN-QM for briefer outcome monitoring Custom specific versions of the above for a given program, site or study People currently working on adaptations for Native Americans, Spanish speakers and American Sign Language

Computer Generated Reports Validity reports to identify areas for clarificaiton and potential problems Text based Personal Feedback Reports (PFR) to support MET/CBT Text based GAIN-Q Referral and Recommendation Summary (GRRS) to support preliminary diagnosis and placement Detailed Individual Clinical Profile (ICP) to support more detailed diagnosis, placement, and treatment planning Government Performance and Results Act (GPRA) reporting requirements report Other site specific reports

GAIN Referral and Recommendation Summary (GRRS) General –Computer Generate Text Narrative –Prompts to check or add text –Gives symptoms to support major diagnosis and insurance claims –Quotes clients Presenting Concerns Five Axis DSM-IV/ICD-9 Diagnoses Evaluation Procedure Substance Use Diagnoses and Treatment History Level of Care and Service Needs by ASAM Placement Criteria Summary Recommendations

Detailed Individual Clinical Profile (ICP) Five Axis DSM-IV Diagnosis I.Substance use disorders, major depression, generalized anxiety, ADHD, CD, and pathological gambling to criteria, screening for mood/anxiety disorders, suicide risk, traumatic distress II.Screening for personality disorders by cluster III.Lifetime history by ICD-9 area and check for common drug-health interactions IV.Traumatic victimization, check for major axis IV bio-psycho-social stressors, and checks for other high-stress events V.Past year and Past 90 day staff ratings for GAF, SOFAS, GARF ASAM PPC2-R Placement –Text statements on diagnosis –Red flag statements on six dimensions (intoxication/withdrawal, biomedical, psychological, relapse potential, treatment readiness, environment) –Scale summaries of problems –Current prevalence and utilization summary

Individual Clinical Profile- Continued Treatment Planning –Client and staff urgency ratings by section –List of things the client wants –Other things typically required by agency or regulation Demographics –Site, staff and client identifiers –Administration information –Demographics –Appearance –Housing situation –Prior treatment –Current involvement in other systems –Staff notes

Training and Quality Assurance Model National Training of Trainers and Local Training Covers administration, scoring, training, quality assurance, data entry set up Includes providing feedback on up to four audio tapes Includes technical assistance installing computer applications Part of a multi-level certification process with continuing education credits in substance abuse counseling, social work, probation, and gambling Certified trainers are able to train, do quality assurance and certify local staff and have on-going access to technical assistance Highest level of trainers certified to help train other agencies/trainers Follow-up technical assistance with local MIS person to help set up and administer

Key Methodological Findings

NIAAA/NIDA Other Grantees CSATs Adolescent Treatment Program Grantees and Collaborators CSAT Cannabis Youth Treatment (CYT) Adolescent Treatment Model (ATM) Strengthening Communities for Youth (SCY) Adolescent Residential Treatment (ART) Effective Adolescent Treatment (EAT) Other CSAT Grantees Other Collaborators RWJF Reclaiming Futures Program RWJF Other RWJF Grantees

Test - Retest We did a test-retest study of the days of use and lifetime marijuana abuse/dependence symptoms over 48 hours or less with 210 adolescent outpatients in CYT. They reported consistent but increasing numbers of –abuse/dependence symptoms (r=.73, 4.6 vs. 5.3 lifetime), –days of marijuana use (r=.74, 31 vs. 34 days) and –days of alcohol use (r=.74, 6 vs. 7 days). Lifetime marijuana abuse/dependence symptoms were internally consistent (Cronbachs alpha=.82). Lifetime marijuana dependence diagnosis was consistent though rising in the second interview (Kappa=.55, 40% vs. 44% lifetime dependence).

Validation To Urine Testing Higher self reported marijuana use than 573 on-site urine tests (83% vs. 76%), with 5% false negative (kappa=.81) Higher self reported marijuana use than 74 quantitative tests (82% vs. 50%), with 3% false negative (kappa=.90) Higher self reported rates of other drugs than laboratory urine tests and breathalyzer tests for alcohol Currently working on predicting false positives and negatives based on self report, validity checks (creatinine, ph., specific gravity), and time from sample to testing

Validation To Collateral Measures Adolescents were more likely than family members or other collaterals to report a greater number of days of any substance use (39 vs. 31 days, t (527) =7.0, p<.001) and cannabis use (37 vs. 30, t (505) =6.0, p<.001) during the past 90 days. They reported slightly fewer days of alcohol use (7 vs. 8, t (505) =-2.2, p<.05) and about the same number of abuse/dependence symptoms of abuse/dependence during the past month (2.4 vs. 2.6 of 11 symptoms, t (594) =-1.6, n.s.d.), past year (4.6 vs. 4.6 symptoms, t (594) =0.1 n.s.d.), and lifetime (5.1 vs. 5.2 symptoms, t (594) =-0.9, n.s.d). main symptom counts (e.g, internal distress, external distress, conduct disorder, aggression) from the GAIN- CAF and CBCL found that similar scales were correlated around.6

Validation To Blind Psychiatric Diagnosis GAIN has also been found to accurately predict diagnoses of co-occurring psychiatric disorders that were made by independent staff blind to GAIN findings including –ADHD (kappa = 1.00), –Mood Disorders (kappa = 0.85), –Conduct Disorder or Oppositional Defiant Disorder (kappa = 0.82), –Adjustment Disorder (kappa = 0.69), and –No other diagnosis (kappa = 0.91) Source: Shane, Jasiukaitis, & Green, 2003

Combining Adolescent and Collateral Symptoms Significantly Increases the Total Number of Symptoms Endorsed

Evaluating the Effects of Treatment Short Term Outcome Stability Difference between average of early (3-6) and latter (9-12) follow-up interviews Treatment Outcome Difference between intake and average of all short term follow-ups (3-12) Long Term Stability Difference between average of short term follow-ups (3-12) and long term follow-up (30) Source: Dennis et al, under review, forthcoming Month Z-Score

Importance of Multiple Measures Over 98% of CYT treatments completed

Adolescent Recovery Pattern Over 12 Mon.s Source: Cannabis Youth Treatment (CYT) study

Comparative Clinical Characteristics of 2968 Clients from 61 Treatment Units Adolescent Inpatient/Therapeutic Community Adolescent Outpatient/IOP Adult Outpatient/IOP/OP Methadone Treatment Adult Inpatient/Therapeutic Community Oakland, CA Shiprock, NMLos Angeles, CA Phoenix/Tempe, AZ Tucson, AZ Miami, FL St. Petersburg, FL Cantonsville, MD Baltimore, MD New York, NY Chicago, IL Peoria, IL Maryville, IL Philadelphia, PA Bloomington, IL Farmington, CT

Hypothesized Structure of the GAINs Psychopathology Measures * Main scales have alpha over.85, subscales over.7

Confirmatory Factor Analysis (CFA) Comparative Fit Index:.974 Root Mean Square Error of Approximation: Internal.27 HSTI.67 DSI.77 ASI.47 TSI.51 External.68 CDI.83 IAI.60 HII.25 Crime/Violence.55 DCI.62 ICI.62 PCI.39 GCTI.55 SA Problems.78 SDIY.51 SAIY.64 SIIY.54 SSI.54 General Severity.50 ri re rv rs Comparative Fit Index:.97 vs.98 Parsimony Ratio:.80 vs.70 CFI x PR:.78 vs.68 Root Mean Square Error of Approximation:.04 vs.04 Invariant vs Variant Across Age and Level of Care

Psychometrics The Hypothesized Psychometric Structure of the GAINs Psychopathology Measures was replicated across age and level of care subgroups in terms of: the internal consistency of the measures convergent and divergent validity of their loading on the four hypothesized factors the hypothesized structure plus two additional cross loadings was confirmed as the best structure the solution was invariant across age and level of care

General Severity 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Low 52%20%54%33% Medium 33%34%26%30% High 15%46%20%38% Adol OP (n=1081) Adol Resd (n=1127) Adult OP (n=219) Adult Resd (n=413)

Substance Problems (abuse, dependence, substance induced problems) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Low 59%21%24%10% Medium 24%27%32%23% High 17%52%44%67% Adol OPAdol ResdAdult OPAdult Resd

Internal Distress (Somatic, Depression, Suicide, Anxiety, Trauma) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Low 55%28%40%19% Medium 32%39%32%33% High 13%33%28%48% Adol OPAdol ResdAdult OPAdult Resd

Behavior Complexity (AD,HD, ADHD, CD) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Low 36%18%62%46% Medium 38%31%20%19% High 26%51%18%35% Adol OPAdol ResdAdult OPAdult Resd

Crime/Violence (property, interpersonal and drug related crime, oral & physical aggression) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Low 40%22%70%56% Medium 37%32%19%27% High 23%46%11%17% Adol OPAdol ResdAdult OPAdult Resd

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Low (n=150)Moderate (n=158)High (n=216) No Crime 1-2 Crimes 3+ Crimes X2(4)=24.56, p<.001 CVI can predict Criminal Activity 30 Months Latter Odds of committing 3+ crime 4 times higher Source: White (2003)

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Low (n=150)Moderate (n=158)High (n=216) No crime Incarcerated Substance Use only Non-violent crime Violent crime X2(8)=18.36, p<.05 CVI Predict Type of Crime 30 Months Latter Odds of committing violent crime 4.5 times higher Source: White (2003)

Global Appraisal of Individual Needs- Quick (GAIN-Q) Designed to identify those in need of referral for a more detailed assessment on substance use and/or mental health problems First used in a needs assessment for Macon County (IL) Court Services (Titus & Godley, 2000) -- screening of the adolescent probation population Currently being used in SCY, RWJF and several individual projects

Description of the GAIN-QS version 2 Designed to be a shorter more general assessment for use with indicated populations (e.g., student or employee assistance programs, juvenile or criminal justice) or needs assessment. 10 pages in length (9 content, 1 case disposition) Interviewer- or self-administered in 15 to 20 minutes Eight sections - Background, General Factors, Sources of Stress, Physical Health, Emotional Health, Behavioral Health, Substance-Related Issues, End First four sections are background and formative indices of factors related to behavioral health problems Total score on 99 yes/no items, that are also divided into four scales and 12 subscales

Substance Abuse (SA) and Mental Health (MH) Needs in Adolescent Probation Source: Titus & Godley 2001

Quick GAIN Indices Total Symptom Severity Index (TSSI – 99 items) General Life Problem Index (GLPI – 50 items) General Factors Index (GFI- 16 items) Sources of Stress Index (SOSI - 20 items) Health Distress Index (HDI – 14 items) Internal Behavior Index (IBI – 17 items) Depression Symptom Index (DSI-5 items) Suicide Risk Index (SRI-5 items) Anxiety Symptom Index (ASI-7 items) External Behavior Index (EBI 16 items) Attention Deficit/Hyperactivity Disorder Index (ADHDI-6) Conduct Disorder/Aggression Index (CDAI-6) General Crime Index (GCI-4) Substance Problems Index (SPI –16 items) Substance Use & Abuse Index (SUAI-9 items) Substance Dependence Index (SDI-7 items)

QS Scales by Level of Care Source: Approximation from ATM data QS depression Symptom index QS Suicide Risk Index QS Anxiety Symptom index QS Internal Behavior Index QS Attention-Hyperactivity Disorder Index QS Conduct Disorder- Aggression Index QS General Crime Index QS External Behavior Index QS Substance Use and Abuse Substance Dependence Index Substance Problem Index TC (n=288)STR (n=604)OP/IOP (n=513)

QS Scales by Gender QS depression Symptom index QS Suicide Risk Index QS Anxiety Symptom index QS Internal Behavior Index QS Attention-Hyperactivity Disorder Index QS Conduct Disorder- Aggression Index QS General Crime Index QS External Behavior Index QS Substance Use and Abuse Substance Dependence Index Substance Problem Index Male (n=935) Female (n=333) Source: Approximation from ATM data

Other Features HIPAA compliant computer applications for data entry or computer assisted interviewing are in development and testing Change be imported into the GAIN for a full assessment Has days and times questions to support analysis of change Has service utilization questions Addition of other outcomes, service utilization module, and substance abuse skip out for non users Referral and Recommendation Summary Report Supplemental Reasons for Quitting module and Personal Feedback Report to support brief interventions with substance users using MET/CBT5

Key Methodological Work Underway ASAM placement recommendations based on expert and statistical models Identification of multi-problem clusters or Code types Modeling Change over time in relations to the treatment hinge and the cycle of relapse, treatment re- entry and recovery Propensity score models to predict outcomes and serve as a synthetic average treatment comparison group Clusters or Code Type labels based on above Economic analysis of costs, cost-effectiveness and benefit costs

Can be used to Measure Changes in Cost to Society $0 $1,000 $2,000 $3,000 $4,000 $5,000 $6, Months from Intake UCHC, Farmington, CT (-24%, -44%) PAR, St. Petersburg, FL (-22%, -49%) CHS, Madison Co., IL (-8%, -51%) CHOP, Philadelphia, PA (+18%, -34%) Source: French et al, 2003

Measuring Improved Adherence to Continuing Care after Residential Treatment Source: Godley et al 2002 ACC * p<.05 0% 10% 20% 30% 40%50%60%70%80%90% 100% WeeklyTx Weekly 12 step meetings Regular urine tests Contact w/ probation/school Follow up on referrals* 0% 10% 20% 30% 40%50%60%70%80%90% 100% Relapse prevention* Communication skills training* Problem solving component* Meet with parents 1-2x month* Weekly telephone contact* Referrals to other services* Discuss probation/school compliance* Adherence: Meets 8/12 criteria* UCC

Reducing Relapse: Marijuana Source: Godley et al 2002 Days to First Alcohol Use (p<.05) Percent Remaining Abstinent UCC ACC

Other Major Methods Studies Underway Internal consistency and norms by age and level of care on website already Reliability comparisons being done in CYT and ERI Validation of self reported use to on-site urine and saliva, Emit and quantitative urine tests, as well as collaterals Exploratory and confirmatory factor analyses done across studies, populations and levels of care Multiple case mix adjustments being tested for comparing programs Prediction of blind psychiatric diagnosis Comparison with records in AAP, CYT, ATM Comparisons with other existing measures (e.g., ARCQ, BAC, Barclay ADHD scale, CBCL, DIS, DOTS-R, FES, FFS, Form 90, Jessors religiosity scale, MMPI, PDQ, RFQ, PPS, Reasons for Quitting, SCIDII, SCL, SM, TLFB, Tolans Parenting Practices measure,Tower of Hanoi, TTS, WAI, WISC-R Digit Span) and protocol or study specific measures (e.g., adherence, discharge, follow-up log, service contact logs)

Validity Checks Currently Available Staff ratings of understanding, misrepresentation, appearance/behaviors during assessment, and context Consistency Reports Counts of missing/refused items Out of normative responses on time, key items Additional Scales in the Works Inconsistency scale Endorsing rare items (faking bad/general severity) Not endorsing common items (faking good/a typical profile)

GAIN/ABS just part of a Trans-Enterprise MIS Host MIS Mgmt Reports Service Logs Appt Tracking Host Acct Sys Host Lab School MIS Welfar e MIS JJS MIS Evaluato r or Data Manager GRL, Other Data Assessment Building System: GAIN, Screener And Other Measures Cross Site Evaluation

Contact Information Michael L. Dennis, Ph.D. Lighthouse Institute, Chestnut Health Systems 720 West Chestnut, Bloomington, IL Phone: (309) , Fax: (309) Website: GAIN Training Coordinator: Michelle White at or