1. Information for Effective Decision-Making: A Multi-Level Approach Nate Israel, PhD nathaniel.israel@sfdph.org

2. Context / Assumptions The system is full of apparent tensions: cost vs quality, managed care vs individual control, competition for scarce resources The system must respond to the needs of both client and political stakeholders No single person or structure alone can adapt the system quickly enough to either political or client needs to both survive and markedly improve care

3. Surviving and Thriving Distributing the responsibility for adaptation is necessary for client welfare and system sustainability Typical top-down models of change often have short-term success at the cost of long term failure to adapt and short-term infighting Reducing in-fighting and increasing the uptake of new behaviors requires new tools

4. Tools for Success The Good News: We’ve spent the past 3 years developing these tools Automated reports have been built (and are being replicated in NetSmart) to facilitate data- based discussions of what works and what can be improved (information for culture change) As-needed reports are now available to address any clinical or functional issue we measure (information for stakeholder queries)

5. Feedback System: Goal To allow persons at each level of the system to use the same data to identify and solve problems at their level of responsibility To allow the system to identify the seriousness of specific issues and the magnitude of response needed to address the issue To allow the system to identify successes to celebrate and replicate

6. Client Needs and Strengths: Initial Needs / Strengths and Change over Time Clinician Needs and Strengths: Client level effectiveness, Cross- client effectiveness, Effectiveness vs Agency and System averages Supervisor Needs and Strengths: Client level effectiveness, Effectiveness of Clinical Supervisees, Relative Strengths by Domain of Strength and Need Program Needs and Strengths: Particular Needs of Clients at Entry; Change in Needs over Time (total change and change by item); Compliance and client Flow Characteristics (dis-engagement rate, time from entry to treatment, number of openings and closing) System Needs and Strengths: Program-by-program profile of Client Needs over Time; System Flow characteristics for individual Programs, Programs within a Level of Care, and the entire System; As-needed reports on any measured aspect of Clinical Symptoms or Functioning CANS Feedback System : Multi-Level Information for Effective Decision-Making

7. Communication Structure Examples: Client-Level Clinical Reports Program-Level Reports System-Level Reports Note: This overview is representative of the types of reports available, but is not an exhaustive description of the reports available.

8. Clinical Reports: Goal To allow clinicians to quickly identify progress in goal achievement at the client and caseload level To allow clinicians to quickly identify how their performance relates to that of other clinicians in the same agency

9. Clinical Reports 1) Change in Syndromes / Functioning over Time

10. Clinical Reports 2) Client and Caseload Level Change over Time

11. Program Level Reports: Goal To allow supervisors to identify clinical successes and issues at the clinician and team level To allow program directors to identify clinical successes and issues at each level of the program (clinician, supervisor, program) To allow program directors to identify the unique needs of their client population and the agency’s effectiveness on meeting each need To allow programs to learn from each other’s successes in effectively meeting clients’ needs

12. Program-Level Reports RU-level change over time

13. Program-Level Reports Agency Clinical Formulation Program / Agency Initial Behavioral / Emotional Needs, Functional Impairments Expected changes at Re-AssessmentExpected changes at DischargeDescription of Change process MST BE/ Needs: 1. Oppositional 2. Depression 3. Anger Control 4. Judgment 5. Delinquency 6. Impulsive / Hyperactive Functional Impairments: 1. Legal 2. School Achievement 3. School Behavior 4. School Attendance BE/ Needs: 1. Less Oppositional Bhvr. 2. Less Delinquent Bhvr. 3. 4. 5. 6. Functional Impairments: 1. Fewer Legal problems (JJ) 2. Improved School Behavior 3. 4. BE/ Needs: 1. Less Oppositional Bhvr. 2. Improved Anger Control 3. 4. 5. 6. Functional Impairments: 1. Fewer Legal problems (JJ) 2. Improved School Behavior 3. Improved School Achiev. 4. Therapists work with the family and youth to reduce problem behaviors, eventually leading to fewer symptoms, less dangerous behavior and better school achievement.

14. Program-Level Reports Agency Clinical Formulation Over Time

15. Program-Level Reports Agency Clinical Formulation: Summary Rating

16. System Level Reports System Flow Reports ACF / Clinical Effectiveness Reports Need / Issue Based Reports

17. System Flow Reports: Goal The most basic goal of a system flow reports is to identify the rate at which clients move through the system (Contract Performance Indicators) Understanding flow at each level of care and program within a level of care allows you to identify system blockages These analyses can also allow you to better understand disparities in access Last, these analyses allow you to identify patterns of movement that may indicate effective/ineffective care

18. System Flow Reports: LOC

19. System Flow Reports Typically look by RU Can then calculate number of new clients, carryover clients, closed clients by quarter Allows us to understand flow Also allows us to monitor blockages in flow

20. ACF Reports: Goal At the System level, these reports allow you to identify : – The effectiveness of the system in addressing client needs – The most prevalent conditions for treatment – Key partners needed to sustain change over time

21. ACF Reports: System Level

22. Need / Issue Based Reports Many needs for information arise unpredictably We can rapidly create brief reports on any clinical or functional issue we track These reports can provide both data on the issue at hand, and contextual data on the persons affected by the issue, which may help us better understand the need

23. Need / Issue Based Reports Key CANS data:

24. Need / Issue Based Reports Context: “Of those children and youth who have serious school attendance problems, 42% also had recent contact with the legal system because of delinquent behavior. Nearly twenty percent (17.8%) of serious truants were involved in criminal behavior (non-status offenses) during the past month. These children and youth may also have experienced neglectful or abusive child-rearing contexts. Over 20% of serious truants reportedly experienced parental neglect; 16% experienced parental physical abuse.”

25. Summary - Strengths We currently have a system of reports useful to describe our children and youth on critical clinical, functional, and service dimensions This system allows us to use similarly structured data at each level of the system to align system improvement with client needs The system also allows us to respond in a structured way to unpredictable or sudden needs and demands