1. Introduction to the Child & Adolescent Needs and Strengths Assessment Tim Connor, MS, MA Mental Health Evaluation Specialist for the Wisconsin Department of Health Services Dave Minden, PhD, Clinical Psychologist Moderated by: Kathy Markeland, WAFCA 6/8/20161

2. Webinar Overview Light Housekeeping You need to dial into the teleconference line to hear the audio portion of this presentation. 1-866-740-1260, Access code: 4200950 I will field questions through the “chat” feature online and we will address them during the last half hour of our presentation. We are recording this event. You will receive a copy of the Power Point presentation after today’s event. 6/8/20162

3. Webinar Overview What is the context for this conversation? What is CANS? Communimetric vs. Psychometric approach CANS Scoring DCF Draft Tool Administration/training/data collection 6/8/20163

4. Webinar Overview Evaluation and Output Other States CANS in Wisconsin What are the deficits/criticisms of CANS? Validity/Reliability of CANS LSS Clinical Story – Choosing CANS and Building CANS Culture 6/8/20164

5. Why are we talking about CANS? 2009-11 Biennial Budget = Wisconsin initiated a number of changes to the child welfare system including a commitment to move to a single statewide standardized child assessment tool. DCF, in consultation with stakeholders, has selected the Child & Adolescent Needs and Strengths (CANS) Assessment. 6/8/20165

6. Why are we talking about CANS? DCF is now in the process of "Wisconsinizing" the tool in preparation for statewide implementation in January 2011. Anticipate that the DCF CANS tool will be piloted in BMCW as early as July 2010. Possible that other agencies/counties will be invited to “pilot” as well. 6/8/20166

7. Why are we talking about CANS? In the Wisconsin child welfare system, CANS will be a significant part of: – Determining the “level of care” required for a specific child. – Measuring a child’s progress while in care and making redeterminations about the “level of care” required. 6/8/20167

8. Why are we talking about CANS? CANS data will also likely contribute to: – Outcome monitoring of programs and system wide (including advising performance-based measures within provider contracts) – Quality improvement efforts both within programs and throughout the system. 6/8/20168

9. What is CANS? The CANS is an information integration tool. The purpose of the CANS is to accurately represent the shared vision of the child serving system, which includes the child, family and all service providers. The CANS is intended to facilitate effective communication of this shared vision for use at all levels of the system. 6/8/20169

10. What is CANS? Designed for use at three levels: – individual child and family – program – system of care Provides a structured assessment of children along a set of dimensions relevant to service planning and decision-making. Provides information regarding the child and family's service needs for use during system planning and/or quality assurance monitoring. 6/8/201610

11. The CANS originated from the Childhood Severity of Psychiatric Illness (CSPI) tool, which assessed the appropriate use of expensive mental health service interventions. CANS ratings are designed to lead directly into treatment planning. All the CANS items can be used to determine the appropriate level of care for a child. CANS has an algorithm for determining the appropriate level of care needed for a child. What is CANS? 6/8/201611

12. What is CANS? Since its primary purpose is communication, the CANS is designed based on communication theory rather than the psychometric theories that have influenced most measurement development. Following slides compare “communimetric” and “psychometric” approach to assessment. 6/8/201612

13. Communimetrics Approach to Assessment Communication value of tool is top priority – communication from family to provider to service plan Content is flexible and must be meaningful to the service delivery process Items selected on clinical rather than statistical criteria “Just enough information” approach to instrument design Trust in the reliability of the responses and expertise of the rater Easy to use and results are actionable

14. Psychometric Approach to Assessment Precise measurement is top priority Scientific replication of measurement Focus on reliability and validity Multiple similar items to test reliability of responses Sophisticated scale scoring Interpreting results for service planning not always straightforward

15. Inputs and Outputs of Assessment

16. Six Characteristics of a Communimetric Tool Item-level tool to provide detail for service planning Item ratings translate immediately into action levels It is about the child not about the service Consider culture and development Primarily a descriptive tool—it is about the ‘what’ not about the ‘why’ The 30-day window is to remind us to keep assessments relevant and ‘fresh’ 6/8/201616

17. Scoring Needs: 0 - No Need 1 - Watch/Prevent 2 - Act 3 - Act Immediately/Intensively Strengths: 0 - Centerpiece 1 - Useful 2 - Potential 3 - None identified 6/8/201617

18. DCF Draft Modules Trauma (sexual abuse, adjustment) Functioning (school, juvenile justice, developmental, medical) Emotional/Behavioral Needs Youth Strengths Acculturation Risk Behavior (runaway) Caregiver strengths and needs – Permanency plan caregiver – Current caregiver 6/8/201618

19. Other Versions of the CANS CANS-Juvenile Justice CANS-Early Childhood (ages 0-3) CANS-Trauma Exposure and Adaptation CANS-Developmental Disabilities CANS-Autism CANS-Sexual Development CANS-Mental Health ANSA (Adult Needs and Strengths Assess.) FAST (Family Advocacy and Support Tool)

20. Administering CANS Method of administration: Instrument is rated by the child’s provider based on information collected during the assessment process. Frequency of administration: The CANS can be completed as often as needed. After the initial assessment at the time of enrollment, an updated assessment is recommended at least every 3 to 6 months. Burden: It takes 15 minutes to complete 45 items on the CANS. 6/8/201620

21. Professional knowledge in the field required; at least a Bachelor’s degree recommended. – Raters need mental health expertise to rate mental health items. 1 ½ - 2 days of training recommended. Must successfully complete test vignette to become certified. Raters maintain the reliability of their ratings by annually completing vignettes provided by the author. Training Requirements 6/8/201621

22. Collecting CANS Data The CANS provides guidance on assessment content and a conceptual approach. – But does not take the place of provider expertise Provider-reliant tool – Expertise is assumed – Some interview prompts available……….. – But focus is on organizing assessment info. to improve communication and gain consensus

23. Data system needed if program- and system- level uses intended At least 3 web-based data systems available from private groups – Objective Arts – Communimetrics Group – Polaris Systems offer a mix of data entry, case management, system management, evaluation, and training functions CANS Data Systems 6/8/201623

24. Scoring: Scale/domain scores can be calculated with a mean of items multiplied by 10. Outcome comparisons are best made at the item level and subscale level (using Total scores loses too much data). CANS has an algorithm for determining the appropriate level of care needed for a child. Evaluation and Output 6/8/201624

25. Changing Practice in Illinois DCFS Problem: increasing use of inpatient and increasing lengths of stay Dr. John Lyons hired to evaluate inpatient placement decisions Children’s Severity of Psychiatric Illness (CSPI) tool developed to verify decisions – Precursor to the CANS

26. Changing Practice in Illinois DCFS Results: – Accuracy of hospital placements increased – Inappropriate placements of African-American and Hispanic children dropped 15-20% – Reduction by 1/3 of children and youth placed in residential treatment in child welfare

27. Family & YouthProgramSystem Decision Support Service Planning Effective practices EBP’s Eligibility Step-down Resource Management Right-sizing Outcome Monitoring Service Transitions & Celebrations EvaluationProvider Profiles Performance/ Contracting Quality Improvement Case Management Integrated Care Supervision CQI/QA Accreditation Program Redesign Transformation Business Model Design Total Clinical Outcomes Management (TCOM) 6/8/201627

28. Current Status of CANS Training for ISP/CST’s Balancing self-training web modules with in- person training Increasing guidance and instructions in written training materials Looking for methods to increase training capacity Integration of “Super Users” into training

29. Integration of CANS with ISP/CST’s Merged instructional manual with interview prompts Incorporated prompts and ratings into existing assessment paperwork Medicaid approved CANS for pre-approval required for in-home therapy billing Integrated ISP/CST CANS paperwork with CCS & targeted case mgmt. requirements

30. Current and Future Status of CANS Use in Wisconsin 45 Integrated Service Programs and Coordinated Service Teams trained – 230 certified trainers and raters Lutheran Social Services implementing Department of Child and Family Services planning for future implementation in foster care system Implications for cross-system applications

31. Deficits/Criticism? Requires consensus and is ‘equalizing’, meaning no ‘expert’ to solve disagreements Training required to use, and recertification yearly is recommended for accuracy Fairly brief, so may miss some level of detail 6/8/201631

32. CANS Validity Validity: Studies have compared the CANS to results from other instruments and clinical judgment and found the CANS to be valid. See manual and literature for details. 6/8/201632

33. CANS Reliability Reliability: Reliability studies have demonstrated that the CANS is reliable at the item level. Training and certification is required for the use of the CANS and the recommended minimum for certification is a reliability of 0.70 using an intraclass correlation coefficient on a test vignette. Average reliability after training is approximately 0.80. Reliability on case record reviews has been demonstrated to be 0.85 while reliability with live interview strategies is above 0.90. 6/8/201633

34. The LSS Clinical Story: History and Current Plan Pilot training, a few staff attended training Utility: achieved LSS’ initial CANS involvement Some staff gave CANS impressions, concerns o Questions: Versions? Administration time length? (longer than Achenbach, others?) o Practice time to competent/efficient administration?

35. Decision-making on CYF going forward with CANS Based on: Focusing tool to match clients/services Fine-tuned program admission/discharge criteria Aggregation of data across clients within program, across programs within service Outcome reporting across iii above Explore/discover/report/develop programs for unmet needs of clients/families Adaptability of CANS as needs/new questions arise Universality: many programs/services within state and nationally using CANS Data/research/algorithms from broad, statewide use

36. LSS: Next Steps Training of September 2009 40 staff from several programs trained Good: 39 immediately certified Poor: general feeling that training was underwhelming Homme staff organized around CANS and are using it well in spite of limited training experience Other staff ‘not-yet-organized’ around CANS and use/understanding is hit or miss Dave Minden offering ongoing consultation to try to obviate not-yet-organized problems

37. LSS: Future Steps Training in April 2010 by Dr. John Lyons, CANS developer 150 Staff in varied CY&F programs to attend Creating CANS culture Parallel with state

38. LSS: And Beyond… Further support – Trainers day April 2010 – Creating in-house training model – Ongoing training of new staff – Clinical support of CANS use Conference call monthly Staffings organized around CANS – Software for recording and reporting

39. Questions? Phone lines now open for questions If you would like to keep your phone muted press “*6” (to unmute press “*7”) You may continue to send questions in via chat. 6/8/201639

40. Additional Resources Praed Foundation www.praedfoundation.org www.praedfoundation.org 6/8/201640

41. Thank You Following this webinar, you will be asked to complete a brief evaluation for continuing education credit purposes and to help us develop future trainings. If you did not log into this webinar, but participated as part of a group at your agency, a separate evaluation form will be emailed to you. You will receive a copy of today’s Power Point presentation via email. CEUs will follow via email within the next two weeks. 6/8/201641