1. Introduction to Outcomes Informed Care

2. What is “Outcomes Informed Care”?  Routine use of patient self report outcome and therapeutic alliance questionnaires to inform the treatment process.  Questionnaires are administered a frequent intervals throughout an episode of care.  Clinicians given access to continuous feedback on patients improvement as measured by the outcome questionnaires.

3. Who benefits?  Strong evidence from controlled studies and real world applications that patients benefit.  Higher probability of improvement  Fewer treatment failures  Clinicians  Increasing market demand for clinicians with strong practice based evidence of effectiveness  Employers & payers  Greater productivity gains  Higher “return on investment”

4. What about the research?  Evidence of patient benefit from use of questionnaires and clinician feedback: References 1 - 7  Evidence for role of common factors: References 8-14  Evidence for importance of the clinician: References 15-25  Evidence for importance of working alliance: References 26-32

5. Tell me about the questionnaires  Items written to 4 th grade reading level  Simple to understand frequency anchors  Never, Hardly Ever, Sometimes, Often, Very Often  Common sentence structure aids rapid completion How often in the past two weeks did you …feel unhappy or sad? …have little or no energy?

7. Reliability & validity  Reliability  Coefficient alpha =>.90  Validity  Strong correlations with other measures of global distress  Correlations of Concurrent Validity  Beck Depression Inventory =.78  PHQ9=.81

8. What is global distress?  All measures commonly used in mental health research appear to load correlate strong with a common factor, common called “global distress”  Global Factor includes items assessing:  Symptoms of depression & anxiety  Attention and concentration problems  Family and interpersonal relations  Work place productivity & functionality

9. Item Response Theory  Permits us to evaluate how well items works for clients at varying levels of distress  Identify items sensitive for clients with very high levels of global distress scores  Thoughts of suicide; Worthless  Outpatient questionnaires ideally targeted for moderate to severe cases with a mix of items

10. Client registration form  Completed at start of treatment  Used to collect case mix variables such as:  Age  Prior treatment history  Sex  Language & ethnicity  Diagnosis  Health status  History of physical and/or sexual abuse

11. How is outcome measured?  Treatment outcome is measured as the improvement between intake scores and scores at the last encounter in a treatment episode  Multivariate modeling used to adjust for difference in case mix  Severity Adjusted Effect Size calculated for all patients with intake scores in the clinical range on the outcome questionnaire

12. How will I get feedback?  Clinician’s Decision Support Toolkit  Data available within 24 hours of submission  Summary statistics  Case level data  Graph results, monitor improvement

16. Relationship building is an Evidence Based Practice! Practitioners are encouraged to routinely monitor patients’ responses to the therapy relationship and ongoing treatment. Such monitoring leads to increased opportunities to repair alliance ruptures, improve the relationship, modify technical strategies, and avoid premature termination. - Norcross & Lambert (2006) in Evidence-Based Practices in Mental Health, Norcross, Beutler & Levant (Eds), p. 218

17. Concept of Therapeutic Alliance Three Components:  Tasks: Behaviors and processes within the therapy session that constitute the actual work of therapy  Bonds: The positive interpersonal attachment between therapist and client of mutual trust, confidence, and acceptance  Goals: Objectives of therapy that both client and therapist endorse

18. Therapeutic Alliance Scale Sample items  I felt like we talked about the things that were important to me  I felt like the therapist liked and understood me  I felt the session was helpful.  I felt confident that the therapist and I worked well together  Did you feel that the clinician understood what it was like to be you

19. Alliance Scale Psychometrics  Items are heavily skewed in positive direction.  Scale scores are not normally distributed.  Cannot calculate reliability & validity using parametric statistics that assume normality of distribution  Items are only as “valid” as clinician’s ability to illicit honest and frank responses!

20. Alliance Results Measurement make a difference

21. High praise for clinicians Alliance scores at start of treatment

22. Alliance changes in treatment

23. Alliance change & outcome

24. An outcomes informed clinician…  Uses the best available data on treatment outcomes to inform the treatment for each client/patient  Recognizes the importance of clinician skill in providing effective treatments.  Accepts personal responsibility for evaluating and improving his or her outcomes.

25. What if my outcomes aren’t “good enough”?  Many clinicians experience anxiety at the thought of “being evaluated”  Overall outcomes tend to be very good, with a very large percentage of clinicians demonstrating effectiveness  Clinicians with lower effect sizes at first usually show substantial improvement over time  The result: Clinicians find that outcomes informed care give them a sense of confidence in their ability to help patients.

26. Who do I call for help?  Questions about missing or incorrect data Audra Day; Data Center Coordinator mailto:datacenter@clinical-informatics.com 801 554-4340  Clinician to clinician consultation & help Joanne Cameron, PhD mailto:joanne@clinical-informatics.com 801 739-6268  Technical questions Jeb Brown PhD mailto:jebbrown@clinical-informatics.com

27. References 1.Lueger RJ. 1998. Using feedback on patient progress to predict the outcome of psychotherapy. J Clin Psychol 54:383-93. 2.Lambert MJ, Whipple JL, Smart DW, et al. 2001. The effects of providing therapists with feedback on patient progress during psychotherapy: Are outcomes enhanced? Psychother Res 11(1):49-68. 3.Hannan C, Lambert MJ, Harmon C et al. 2005. A lab test and algorithms for identifying clients at risk for treatment failure. J Clin Psychol 61(2):155-63. 4.Lambert MJ, Harmon C, Slade K et al. 2005. Providing feedback to psychotherapists on their patients progress: Clinical results and practice suggestions J Clin Psychol 61(2):165-74. 5.Harmon C, Hawkins, Lambert MJ et al. 2005. Improving outcomes for poorly responding clients: The use of clinical support tools and feedback to clients. J Clin Psychol 61(2):175-85. 6.Brown GS, Jones DR. 2005. Implementation of a feedback system in a managed care environment: What are patients teaching us? J Clin Psychol 61(2):187-98. Claiborn CD, Goodyear EK. 2005. Feedback in psychotherapy. J Clin Psychol 61(2):209-21.

28. References (continued) 7.Azocar, F., Cuffel, B, McCulloch, J., McCabe, J., Tani, S., Brodey, B. (2007) Monitoring patient improvement and treatment outcomes in managed behavioral healthcare. Journal for Healthcare Quality (March/April) 8.Rosenzweig S. 1936. Some implicit common factors in diverse methods of psychotherapy: “At last the Dodo said, ‘Everybody has won and all must have prizes.’” Am J Orthopsychiatry 6:412-5. 9.Shapiro DA & Shapiro D. 1982. Meta-analysis of comparative therapy outcome studies: A replication and refinement. Psychol Bull 92:581-604. 10.Robinson LA, Berman JS, Neimeyer RA. 1990. Psychotherapy for treatment of depression: A comprehensive review of controlled outcome research. Psychol Bull 108:30-49. 11.Wampold BE, Mondin GW, Moody M, et al. 1997. A meta-analysis of outcome studies comparing bona fide psychotherapies: Empirically, “All must have prizes.” Psychol Bull 122:203-15. 12.Ahn H, Wampold BE. 2001. Where oh where are the specific ingredients? A meta-analysis of component studies in counseling and psychotherapy. J Counsel Psychol 48:251-7.

29. References (continued) 13.Chambless DL, Ollendick TH. 2001. Empirically supported psychological interventions: Controversies and evidence. Annual Rev Psychol 52:685-716. 14. Luborsky, L., Rosenthal, R., Diguer, L., et al. 2002. The dodo bird verdict is alive and well--mostly. J. Psychotherapy Integration Vol 12(1) 32-57 15.Luborsky L, Crits-Christoph P, McLellan T, et al. 1986. Do therapists vary much in their success? Findings from four outcome studies. Am J Orthopsychiatry 56:501-12. 16.Crits-Christoph P, Baranackie K, Kurcias JS, et al. 1991. Meta-analysis of therapist effects in psychotherapy outcome studies. Psychother Res 1:81-91. 17.Crits-Christoph P, Mintz J. 1991. Implications of therapist effects for the design and analysis of comparative studies of psychotherapies. J Consul Clin Psychol 59:20-6. 18.Wampold BE. 1997. Methodological problems in identifying efficacious psychotherapies. Psychother Res 7:21-43. 19.Elkin I. 1999. A major dilemma in psychotherapy outcome research: Disentangling therapists from therapies. Clin Psychol Sci Prac 6:10- 32.

30. References (continued) 20.Wampold BE, Serlin RC. 2000. The consequences of ignoring a nested factor on measures of effect size in analysis of variance designs. Psychol Methods 4:425-33. 21.Huppert JD, Bufka LF, Barlow DH, et al. 2001. Therapists, therapist variables, and cognitive-behavioral therapy outcomes in a multicenter trial for panic disorder. J Consul Clin Psychol 69:747-55. 22.Okiishi J, Lambert MJ, Nielsen SL, et al. 2003. Waiting for supershrink: An empirical analysis of therapist effects. Clin Psychol Psychother 10:361-73. 23.Kim DM, Wampold BE, Bolt DM. 2006. Therapist effects and treatment effects in psychotherapy: Analysis of the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Psychother Res. 16(2): 161-172. 24.McKay, K. M., Imel, Z. E., & Wampold, B. E. (2006). Psychiatrist effects in the psychopharmacological treatment of depression. Journal of Affective Disorders, 92, 287-290.

31. References (continued) 25.Wampold BE, Brown GS. 2005. Estimating variability in outcomes due to the therapist: A naturalistic study of outcomes in managed care. J Consul Clin Psychol. 73(5): 914-923. 26.Bachelor, A., & Horvath, A. (1999). The therapeutic relationship. In M.A. Hubble, B.L. Duncan, and S.D. Miller (eds.). The Heart and Soul of Change: What Works in Therapy. Washington, D.C.: APA Press, 133-178. 27.Blatt, S. J., Zuroff, D.C., Quinlan, D.M., & Pilkonis, P. (1996). Interpersonal factors in brief treatment of depression: Further analyses of the NIMH Treatment of Depression Collaborative Research Program. J Consul Clin Psychol. 64, 162- 171. 28.Bordin, E. S. (1979). The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research, and Practice, 16, 252-260. 29.Burns, D., & Nolen-Hoeksema, S. (1992). Therapeutic empathy and recovery from depression in cognitive-behavioral therapy: A structural equation model. J Consul Clin Psychol. 60, 441-449.

32. References (continued) 30.Connors, GJ, DiClemente, CC., Carroll, KM, et al. 1997 The therapeutic alliance and its relationship to alcoholism treatment participation and outcome. J Consul Clin Psychol, 65(4), 588-598. 31.Horvath, A. O., & Symonds, B. D. (1991). Relation between working alliance and outcome in psychotherapy: A meta-analysis. J Consul Clin Psychol. 38, 139-149. 32.Krupnick, J., Sotsky, SM, Simmens, S et al. (1996) The role of the therapeutic alliance in psychotherapy and pharmacotherapy outcome: Findings in the National Institute of Mental Health Treatment of Depression Collaborative Research Project. J Consul Clin Psychol., 64, 532-539.