1. APS Outcome-Informed Care Clinician Training

2. Overview Importance & Benefits of Outcome-Informed Care APS Outcome-Informed Care Initiative –Goal –Process –Clinician Tool Kit Question & Answer Appendix –Resources –Research

3. What is Outcome-Informed Care? Outcome-Informed Care is the practice and system of providing psychotherapy treatment that is informed by patient-reported treatment outcomes.

4. Importance of Outcome-Informed Care Routine patient feedback to clinicians improves treatment outcomes 26-35 –Identifies patients who are most at risk for treatment failure –Increases identification of clinical risks Suicide: Routine feedback improved detection by 15% The Journal of Crisis Intervention and Suicide Prevention. Vol. 24, No. 2, 2003, pp. 49-55 Substance abuse: Significant reduction in under-reporting with introduction of feedback Joint Commission Journal on Quality and Safety, Vol. 30 (8), August 2004, pp. 448- 454 –Early identification permits proactive work to keep patients engaged in treatment

5. Importance of Outcome-Informed Care Recommended by APA & IOM –APA 2005 EBPP standard: “ongoing monitoring of patient progress and adjustment of treatment as needed are essential to EBPP” (p. 3) –IOM 2006 standard 4.2: “Clinicians and organizations providing services should: –Increase their use of valid and reliable patient questionnaires or other patient-assessment instruments…to assess the progress and outcomes of treatment systematically and reliably –Use measures of processes and outcomes of care to continuously improve the quality of care…” (p. 12)

6. Benefits of Outcome-Informed Care Improves patient outcomes Objective measurement system –National normative database (1996 – present) –Case mix adjustment –Data demonstrates quality of care Provides data/trends not currently available –Collects patient self-reported diagnostic and symptom information –Measures and tracks reported progress –Real-time treatment outcome report

7. APS Outcome-Informed Care Initiative GOAL: Assist clinician in improving member health Tactics: –Educate about outcome-informed care –Provide tools to deliver outcome-informed care Initial Clinician assessment (baseline) Member survey (measures) –Global Distress –Substance abuse –Therapist alliance –Overall health status Online Clinician toolkit (view/track patient feedback/outcomes) –Alert clinician of member elevated risk –Encourage voluntary enrollment in initiative via incentives

8. APS Outcome-Informed Care: Process APS: –Mail pre-printed Provider Assessment & Member Survey with authorization packet –Post patient/member feedback on secure website –Contact clinician if member survey shows elevated risk Office staff: –Administer member survey every session –After clinician review, fax to 866-831-7962 Clinician –Complete initial provider assessment –View outcome data on secure personal webpage (clinician toolkit) within 24 hours view –Discuss with patients; create a culture of feedback within each treatment

9. Provider Assessment Demonstrates outcomes –Baseline for member survey –Case mix variables Data-driven –Tested –Measured User friendly –1 page –8 questions

10. Member Survey Measures –Global Distress Anxiety and depression Interpersonal problems Impaired productivity (work & other daily activities Concerns about health problems Feelings of hopelessness & thoughts of suicide –Substance abuse –Therapist alliance –Overall health status

11. Clinician Toolkit Secure online access to outcome data Summary statistics and case level data –Graph progress for individual cases –Compare results to national benchmarks & meta- analyses of psychotherapy studies Outcome data updated multiples times per day

16. Improved Outcomes: Success! (baseline 2006) 2006 Baseline 2007 9% improvement 2008 22% improvement

17. Question & Answer

18. Appendix

19. Online Toolkit Register or Login here Links to FAQ, Forms & OIC information

20. Online Toolkit: Registration

21. Online Toolkit: Access

22. Appendix: Resources Frequently Asked Questions  www.psychoutcomes.org/APS www.psychoutcomes.org/APS Technical/Data/Web:  admin@psychoutcomes.org admin@psychoutcomes.org  datacenter@clinical-informatics.com datacenter@clinical-informatics.com  801-541-9720 General comments or questions:  oic@apshealthcare.com oic@apshealthcare.com  1.800.305.3720, extension 3320

23. Appendix: Supporting Research 1. Wampold BE. 2001. The great psychotherapy debate: Models, Methods, and Findings. Mahwah NJ: Lawrence Erlbaum Associates. 272 pp.Wampold BE 2. Westen D, Morrison K. 2001. A multidimensional meta-analysis of treatments for depression, panic, and generalized anxiety disorder: An empirical examination of the status of empirically supported therapies. J Consul Clin Psychol 69:875-99.Westen D, Morrison K. 2001. 3. Sapyta J, Riemer M, Bickman L. 2005. Feedback to Clinician: theory, research, and practice. J Clin Psychol 61(2):145-53.Sapyta J, Riemer M, Bickman L. 2005. 4. Hannan C, Lambert MJ, Harmon C et al. 2005. A lab test and algorithms for identifying clients at risk for treatment failure. J Clin Psychol 61(2):155-63.Hannan C, Lambert MJ, Harmon C et al. 2005 5. Lambert MJ, Harmon C, Slade K et al. 2005. Providing feedback to psychoClinicians on their patients progress: Clinical results and practice suggestions J Clin Psychol 61(2):165-74.Lambert MJ, Harmon C, Slade K et al. 2005 6. Harmon C, Hawkins, Lambert MJ et al. 2005. Improving outcomes for poorly responding clients: The use of clinical support tools and feedback to clients. J Clin Psychol 61(2):175-85.Harmon C, Hawkins, Lambert MJ et al. 2005 7. Brown GS, Jones ER. 2005. Implementation of a feedback system in a managed care environment: What are patients teaching us? J Clin Psychol 61(2):187-98.Brown GS, Jones ER. 2005. 8. Miller SD, Duncan BL, Ryan S, et al. 2005. The Partners for Change Outcome Management System. J Clin Psychol 61(2):199-208.Miller SD, Duncan BL, Ryan S, et al. 2005. 9. Claiborn CD, Goodyear EK. 2005. Feedback in psychotherapy. J Clin Psychol 61(2):209-21.Claiborn CD, Goodyear EK. 2005. 10. Brown GS, Burlingame GM, Lambert MJ, et al. 2001. Pushing the quality envelope: A new outcomes management system. Psychiatr Serv 52(7):925-34.Brown GS, Burlingame GM, Lambert MJ, et al. 2001. 11. Lueger RJ. 1998. Using feedback on patient progress to predict the outcome of psychotherapy. J Clin Psychol 54:383-93.Lueger RJ. 1998. 12. Lambert MJ, Whipple JL, Smart DW, et al. 2001. The effects of providing Clinicians with feedback on patient progress during psychotherapy: Are outcomes enhanced? Psychother Res 11(1):49-68.Lambert MJ, Whipple JL, Smart DW, et al. 13. Lambert MJ, Whipple JL, Vermeersch DA, et al. 2002. Enhancing psychotherapy outcomes via providing feedback on client progress: A replication. Clin Psychol Psychother 9:91-103.Lambert MJ, Whipple JL, Vermeersch DA, et al. 14. Whipple JL, Lambert MJ, Vermeersch DA, et al. 2003. Improving the effects of psychotherapy: The use of early identification of treatment failure and problem-solving strategies in routine practice. J Counsel Psychol 50(1):59-68.Whipple JL, Lambert MJ, Vermeersch DA, et al. 15. Lambert MJ, Whipple JL, Hawkins EJ, et al. 2003. Is it time for clinicians to routinely track patient outcome? A meta-analysis. Clin Psychol Sci Prac 10:288-301. 16. Addis ME. 2002. Methods for disseminating research products and increasing evidence-based practice: Promises, obstacles, and future directions. Clin Psychol Sci Prac 9:367-78. 17. Chorpita BF, Yim LM, Donkervoet JC, et al. 2002. Toward large-scale implementation of empirically supported treatments for children: A review and observations by the Hawaii Empirical Basis to Services Task Force. Clin Psychol Sci Prac 9:165-90. 18. Herschell AD, McNeil CB, McNeil DW. 2004. Clinical child psychology’s progress in empirically supported treatments. Clin Psychol Sci Prac 11:267-88. 19. Manderscheid RW, Henderson MJ. 2004. Mental health, United States, 2002 executive summary. Admin Policy Mental Health 32:49-55. 20. Stirman SW, Crits-Christoph P, DeRubeis RJ. 2004. Achieving successful dissemination of empirically supported psychotherapies: A synthesis of dissemination theory. Clin Psychol Sci Prac 11:343-59. 21. Rosenzweig S. 1936. Some implicit common factors in diverse methods of psychotherapy: “At last the Dodo said, ‘Everybody has won and all must have prizes.’” Am J Orthopsychiatry 6:412-5. 22. Shapiro DA, Shapiro D. 1982. Meta-analysis of comparative therapy outcome studies: A replication and refinement. Psychol Bull 92:581-604. 23. Robinson LA, Berman JS, Neimeyer RA. 1990. Psychotherapy for treatment of depression: A comprehensive review of controlled outcome research. Psychol Bull 108:30-49.Robinson LA, Berman JS, Neimeyer RA. 1990.

24. Appendix: Supporting Research 24. Wampold BE, Mondin GW, Moody M, et al. 1997. A meta-analysis of outcome studies comparing bona fide psychotherapies: Empirically, “All must have prizes.” Psychol Bull 122:203-15. 25. Ahn H, Wampold BE. 2001. Where oh where are the specific ingredients? A meta-analysis of component studies in counseling and psychotherapy. J Counsel Psychol 48:251-7. 26. Chambless DL, Ollendick TH. 2001. Empirically supported psychological interventions: Controversies and evidence. Annual Rev Psychol 52:685-716.Chambless DL, Ollendick TH. 2001. 27. Martindale C. 1978. The Clinician-as-fixed-effect fallacy in psychotherapy research. J Consult Clin Psychol 46:1526-30. 28. Luborsky L, Crits-Christoph P, McLellan T, et al. 1986. Do Clinicians vary much in their success? Findings from four outcome studies. Am J Orthopsychiatry 56:501-12. 29. Crits-Christoph P, Baranackie K, Kurcias JS, et al. 1991. Meta-analysis of Clinician effects in psychotherapy outcome studies. Psychother Res 1:81-91. 30. Crits-Christoph P, Mintz J. 1991. Implications of Clinician effects for the design and analysis of comparative studies of psychotherapies. J Consul Clin Psychol 59:20-6.Crits-Christoph P, Mintz J. 1991. 31. Wampold BE. 1997. Methodological problems in identifying efficacious psychotherapies. Psychother Res 7:21-43, 32. Elkin I. 1999. A major dilemma in psychotherapy outcome research: Disentangling Clinicians from therapies. Clin Psychol Sci Prac 6:10- 32. 33. Wampold BE, Serlin RC. 2000. The consequences of ignoring a nested factor on measures of effect size in analysis of variance designs. Psychol Methods 4:425-33. 34. Huppert JD, Bufka LF, Barlow DH, et al. 2001. Clinicians, Clinician variables, and cognitive-behavioral therapy outcomes in a multicenter trial for panic disorder. J Consul Clin Psychol 69:747-55.Huppert JD, Bufka LF, Barlow DH, et al. 2001. 35. Luborsky L, Rosenthal R, Diguer L, et al. 2002. The dodo bird verdict is alive and well—mostly. Clin Psychol Sci Prac 9:2-12. 36. Okiishi J, Lambert MJ, Nielsen SL, et al. 2003. Waiting for supershrink: An empirical analysis of Clinician effects. Clin Psychol Psychother 10:361-73. 37. Brown GS, Jones ER, Lambert MJ, et al. 2005. Identifying highly effective psychoClinicians in a managed care environment. Am J Managed Care 11(8):513-20.Brown GS, Jones ER, Lambert MJ, et al. 2005. 38. Wampold BE, Brown GS. 2005. Estimating Clinician variability: A naturalistic study of outcomes in private practice. J Consul Clin Psychol.73(5): 914-923.Wampold BE, Brown GS. 39. Kim DM, Wampold BE, Bolt DM. 2006. Clinician effects and treatment effects in psychotherapy: Analysis of the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Psychother Res. 16(2):161-172. 40. Matsumoto K, Jones E, Brown GS. 2003. Using clinical informatics to improve outcomes: A new approach to managing behavioral healthcare. J Info Tech Health Care 1(2):135-50. 41. Brown GS, Jones ER, Betts W, et al. 2003. Improving suicide risk assessment in a managed-care environment. Crisis 24(2):49-55. 42. Brown GS, Herman R, Jones ER, et al. 2004. Improving substance abuse assessments in a managed care environment. Joint Commission J Quality Safety 30(8):448-54. 43. Wells MG, Burlingame GM, Lambert MJ, et al. 1996. Conceptualization and measurement of patient change during psychotherapy: Development of the Outcome Questionnaire and Youth Outcome Questionnaire. Psychother 33:275-283. 44. Lambert MJ, Hatfield DR, Vermeersch DA, et al. 2001. Administration and scoring manual for the LSQ (Life Status Questionnaire). East Setauket, NY: American Professional Credentialing Services. 45. Burlingame GM, Jasper BW, Peterson G, et al. 2001. Administration and scoring manual for the YLSQ. East Setauket, NY: American Professional Credentialing Services. 46. Vermeersch DA, Lambert MJ, Burlingame GM. 2002. Outcome Questionnaire: Item sensitivity to change. J Pers Assess 74:242-61.