1 On Track Advanced Topics Getting the Most Out of Your Outcomes Data Eric Hamilton, M.S. Vice President of Clinical Informatics, ValueOptions Jeb Brown, Ph.D. Director, Center for Clinical Informatics

2 Overview 1.What can outcomes data add to my practice? 2.A closer look at the Client Feedback Form 3.What is MyToolkit and what do all those numbers mean? 4.Putting data into practice 5.Discussion

3 What can outcomes data add to my practice? Do you always know which patients struggling in therapy? Do some of your clients stop therapy before you can establish a good relationship? Can you show clients evidence of their progress compared to benchmarks? Can you show prospective clients, colleagues, or managed care companies evidence of your clinical outcomes?

4 How good are clinicians at identifying clients at risk for treatment failure? Lambert and colleagues compared clinician predictions to predictive algorithms using early client self-assessment data Only 3 out of 550 cases predicted to have poor outcome Of the 40 that had a poor outcome, only 1 had been predicted by a clinician Algorithms based on early assessments identified 85% of poor outcome cases – but also identified some that did well Source: Hannan, et al (2005), JCLP Number of cases

5 What do the predictive algorithms mean? Statistical model that compares early client progress to a “typical profile,” adjusting for case characteristics such as intake severity Poor early progress does not “predict” a likelihood of failure – rather it is an indicator of heightened drop-out risk “Off-track” clients who remain engaged generally get good outcomes – it just might take them longer Global Distress Score

6 Client Feedback Form (CFF) - Adult Item Groups Global Distress: 1-10 – Core scale – Sensitive to change over time – Depression, anxiety, social functioning Risk of self-harm: 5 – Risk indicator Substance use: – Risk indicator Work productivity: – Indicator of functioning Therapeutic alliance: – Support tool for therapist Background items: – For case-mix analysis and identification of co- morbidities

7 Tips for Using the CFF Print multiple copies before you need them Have available for the member to complete before the session begins Show you value the data – when giving the CFF and when it is returned Review scored results online before the next session Measure early and measure often!

8 CFF Results: What do all those numbers mean? Access On Track results

9 Outcomes based on the most recent CFF High scores in red

10 You decide which variables to display

11 Results: Key things to remember “Effect Size” is a way of looking at the combined results of cases with 2+ data points, compared to normative data Links at the top of each column take you to more information All scores are presented as means – Range is from zero to four Higher severity results appear in Red GDS = Global distress score (Questions 1-10) Change Score = Difference in the mean GDS score from first assessment to the most recent Benchmark Score = How much better or worse the change score is compared to norms

12 Interpreting Adult Global Distress Scores Scores of 0 to 1.5 – Typical of community (non-treatment) samples Scores of 1.6 to 2.5 – Moderate distress – Only 20% of community samples would be expected to score in this range; about half of individuals seeking treatment score in this range Scores of 2.6 to 4.0 – Severe distress – 25% of individuals seeking MH services score in this range

13 The Trajectory of Change Graph Case is “off-track” compared to the benchmark projection

14 Putting Data Into Practice Most cases don’t require a lot of review of the data, look for the high risk results Talk with your clients about their results, especially when they share something new or surprising An “off-track” cases does not mean a big change is needed – some encouragement to stay engaged may be all that is needed Be sensitive to small changes in the alliance questions If your effect size is low at first, keep measuring and reviewing the data Alliance items are a great place to look for clues for improving effectiveness

15 Questions and Discussion Resources for Questions – Frequently Asked Questions On the web site, near bottom of the ValueOptions page – Technical/Data/Web: to – General comments or questions: to or Call On Track Customer Service