Information for Effective Decision-Making: A Multi-Level Approach Nate Israel, PhD

Context / Assumptions The system is full of apparent tensions: cost vs quality, managed care vs individual control, competition for scarce resources The system must respond to the needs of both client and political stakeholders No single person or structure alone can adapt the system quickly enough to either political or client needs to both survive and markedly improve care

Surviving and Thriving Distributing the responsibility for adaptation is necessary for client welfare and system sustainability Typical top-down models of change often have short-term success at the cost of long term failure to adapt and short-term infighting Reducing in-fighting and increasing the uptake of new behaviors requires new tools

Tools for Success The Good News: We’ve spent the past 3 years developing these tools Automated reports have been built (and are being replicated in NetSmart) to facilitate data- based discussions of what works and what can be improved (information for culture change) As-needed reports are now available to address any clinical or functional issue we measure (information for stakeholder queries)

Feedback System: Goal To allow persons at each level of the system to use the same data to identify and solve problems at their level of responsibility To allow the system to identify the seriousness of specific issues and the magnitude of response needed to address the issue To allow the system to identify successes to celebrate and replicate

Client Needs and Strengths: Initial Needs / Strengths and Change over Time Clinician Needs and Strengths: Client level effectiveness, Cross- client effectiveness, Effectiveness vs Agency and System averages Supervisor Needs and Strengths: Client level effectiveness, Effectiveness of Clinical Supervisees, Relative Strengths by Domain of Strength and Need Program Needs and Strengths: Particular Needs of Clients at Entry; Change in Needs over Time (total change and change by item); Compliance and client Flow Characteristics (dis-engagement rate, time from entry to treatment, number of openings and closing) System Needs and Strengths: Program-by-program profile of Client Needs over Time; System Flow characteristics for individual Programs, Programs within a Level of Care, and the entire System; As-needed reports on any measured aspect of Clinical Symptoms or Functioning CANS Feedback System : Multi-Level Information for Effective Decision-Making

Communication Structure Examples: Client-Level Clinical Reports Program-Level Reports System-Level Reports Note: This overview is representative of the types of reports available, but is not an exhaustive description of the reports available.

Clinical Reports: Goal To allow clinicians to quickly identify progress in goal achievement at the client and caseload level To allow clinicians to quickly identify how their performance relates to that of other clinicians in the same agency

Clinical Reports 1) Change in Syndromes / Functioning over Time

Clinical Reports 2) Client and Caseload Level Change over Time

Program Level Reports: Goal To allow supervisors to identify clinical successes and issues at the clinician and team level To allow program directors to identify clinical successes and issues at each level of the program (clinician, supervisor, program) To allow program directors to identify the unique needs of their client population and the agency’s effectiveness on meeting each need To allow programs to learn from each other’s successes in effectively meeting clients’ needs

Program-Level Reports RU-level change over time

Program-Level Reports Agency Clinical Formulation Program / Agency Initial Behavioral / Emotional Needs, Functional Impairments Expected changes at Re-AssessmentExpected changes at DischargeDescription of Change process MST BE/ Needs: 1. Oppositional 2. Depression 3. Anger Control 4. Judgment 5. Delinquency 6. Impulsive / Hyperactive Functional Impairments: 1. Legal 2. School Achievement 3. School Behavior 4. School Attendance BE/ Needs: 1. Less Oppositional Bhvr. 2. Less Delinquent Bhvr Functional Impairments: 1. Fewer Legal problems (JJ) 2. Improved School Behavior BE/ Needs: 1. Less Oppositional Bhvr. 2. Improved Anger Control Functional Impairments: 1. Fewer Legal problems (JJ) 2. Improved School Behavior 3. Improved School Achiev. 4. Therapists work with the family and youth to reduce problem behaviors, eventually leading to fewer symptoms, less dangerous behavior and better school achievement.

Program-Level Reports Agency Clinical Formulation Over Time

Program-Level Reports Agency Clinical Formulation: Summary Rating

System Level Reports System Flow Reports ACF / Clinical Effectiveness Reports Need / Issue Based Reports

System Flow Reports: Goal The most basic goal of a system flow reports is to identify the rate at which clients move through the system (Contract Performance Indicators) Understanding flow at each level of care and program within a level of care allows you to identify system blockages These analyses can also allow you to better understand disparities in access Last, these analyses allow you to identify patterns of movement that may indicate effective/ineffective care

System Flow Reports: LOC

System Flow Reports Typically look by RU Can then calculate number of new clients, carryover clients, closed clients by quarter Allows us to understand flow Also allows us to monitor blockages in flow

ACF Reports: Goal At the System level, these reports allow you to identify : – The effectiveness of the system in addressing client needs – The most prevalent conditions for treatment – Key partners needed to sustain change over time

ACF Reports: System Level

Need / Issue Based Reports Many needs for information arise unpredictably We can rapidly create brief reports on any clinical or functional issue we track These reports can provide both data on the issue at hand, and contextual data on the persons affected by the issue, which may help us better understand the need

Need / Issue Based Reports Key CANS data:

Need / Issue Based Reports Context: “Of those children and youth who have serious school attendance problems, 42% also had recent contact with the legal system because of delinquent behavior. Nearly twenty percent (17.8%) of serious truants were involved in criminal behavior (non-status offenses) during the past month. These children and youth may also have experienced neglectful or abusive child-rearing contexts. Over 20% of serious truants reportedly experienced parental neglect; 16% experienced parental physical abuse.”

Summary - Strengths We currently have a system of reports useful to describe our children and youth on critical clinical, functional, and service dimensions This system allows us to use similarly structured data at each level of the system to align system improvement with client needs The system also allows us to respond in a structured way to unpredictable or sudden needs and demands