Presentation is loading. Please wait.

Presentation is loading. Please wait.

Using Outcomes in the Quality Improvement Process: Setting System Level Standards Erik R. Stewart, Ph.D. Vice President, System Performance

Similar presentations


Presentation on theme: "Using Outcomes in the Quality Improvement Process: Setting System Level Standards Erik R. Stewart, Ph.D. Vice President, System Performance"— Presentation transcript:

1 Using Outcomes in the Quality Improvement Process: Setting System Level Standards Erik R. Stewart, Ph.D. Vice President, System Performance Renee Kopache, M.S. Recovery Coordinator

2 History/Purpose of Outcomes Ohio Mental Health Outcomes Task Force (OTF) – 1996 –Charge Develop a statewide approach to measuring consumer outcomes in Ohio’s publicly-supported mental health system Vital Signs (1998) Outcomes Implementation Pilot Coordinating Group –Final recommendations were made in 1999.

3 Adult Outcomes Domains Clinical Status –looks at symptoms that a person may experience from their illness and how much they interfere with their daily living. Quality of Life –examines how “good” a person’s life is, and if their needs are being met. Also examines how much control a person has over the events in their life (empowerment).

4 Adult Outcomes Domains Functional Status –This domain identifies how well a person is doing in the community including areas such as work, school and social relationships. Safety & Health –Addresses how a person is doing physically and the amount of freedom they have from psychological harm from self and others.

5 Adult Outcomes: Instruments Adult Consumer Form A –This form is used by consumers with a severe mental illness. (case management) –The survey consists of four parts: Quality of Life, Safety and Health, Symptom Distress and Empowerment.

6 Adult Outcomes: Instruments Adult Provider Form A –consists of two parts: Functional Status and Safety & Health. –Provider’s observations and clinical judgments about the consumer’s social & role functioning, housing status, activities of daily living, criminal justice involvement, harmful behavior and victimization.

7 Adult Outcomes: Instruments Adult Consumer Form B –Consists of three parts: Quality of Life (excluding empowerment, Safety & Health and Symptom Distress. –This shorter version of the Adult A form is used for adult outpatient clients. Providers of non-SMD consumers are not required to complete a provider survey.

8 Youth Outcomes Domains Four primary areas or domains of assessment were selected for Ohio Scales: –Problem severity –Functioning –Hopefulness –Satisfaction with behavioral health services

9 Youth Outcomes: Instruments Ohio Scales Three parallel forms: –Y-form is completed by the youth (self- report for ages 12 and older). –P-form is completed by the youth’s parent (or primary caretaker). –W-form is completed by the youth’s agency worker/case manager.

10 Administration Periods Initial 6 Month 12 Month Annually thereafter (Adult Consumer Form B: Initial and Discharge administrations only)

11 OCO History in Hamilton County Data collection begins – September, 2001 Regular and varied production reporting begins at same time Production performance abysmal, though consistent with other participating areas of State

12 HCCMHB surveys agencies to discern impediments to production

13 Agencies React

14 HCCMHB representatives visit agencies to discern impediments to production

15

16 Impediments (in no particular order): 1) Excuses: POV boxes don’t turn on (batteries) Staff refuse to complete or administer Multi-million $ agency has only 1 printer POV box times out too quickly

17 Impediments (in no particular order): 2) Collection technology: POV data system interface with other agency data system(s)

18 Impediments (in no particular order): 3) Lack of agency/staff “buy-in” We know we do good work We don’t care about consumer outcomes

19 Impediments (in no particular order): 4) Lack of products/information to staff following administrations (initially) Staff frustration with initial software Introduction of ARROW with use limited to POV Immediate red flags printout Weekly reports to staff Consistent supervisor response to problems Use of data in “assessment update printout”

20 Impediments (in no particular order): 5) Diffusion of responsibility (a social phenomenon that occurs in groups of people when responsibility is not explicitly assigned) Request made by QA/Outcomes member to resurrect regular IT/IS staff meetings at HCCMHB. Solicitation made. ONE response. Lack of departmental ownership

21 Impediments (in no particular order): 6) Agency-specific challenges Central Point of Access New administration at agency intake? Paper and pencil administration Program support staff enter data

22 QA Approach Introduced Quarterly indicator measuring Initial Adult Provider compliance added to existing indicators

23 ODMH Introduces Missing Data Reports - Summer, 2003 Hamilton County Performance –23% Adult OR/OE –34% Youth OR/OE

24 October, HCCMHB and Agencies Form PI Sub-committee to Develop Production Logic

25 February, 2004 – Ohio Department of Mental Health Presents State Plan for Upcoming PI Requirements

26 PDCA Cycle Plan Do Check Act

27 April, 2004 – Proposed Logic for OCO Production Measures Shared With Membership

28 Initial Adult Consumer Production Analysis

29 Claims data compiled for quarter under review UCIFirst Svc. DateMin AgeUPIDService Type /10/ CPST /1/ Counseling /2/ CPST /20/ Med Som /25/ CPST

30 Initial Adult Consumer Production Analysis “Pre-quarter period” is determined by subtracting 120 days from first date of service in period UCIFirst Svc. DatePre-Quarter Period /10/20053/12/ /1/20054/3/ /2/20055/5/ /20/20053/22/ /25/20053/27/2005

31 Initial Adult Consumer Production Analysis Review of claims during pre- quarter period is conducted. Those exhibiting claims are removed UCIFirst Svc. DatePre-Quarter PeriodPre-Quarter Claims /10/20053/12/2005No /1/20054/3/2005No /2/20055/5/2005Yes /20/20053/22/2005No /25/20053/27/2005No

32 Initial Adult Consumer Production Analysis Unduplicate list by selecting case with highest order service UCI First Svc. Date Pre-Quarter Period Min AgeUPID Service Type /10/20053/12/ CPST /1/20054/3/ Counseling /20/20053/22/ Med Som /25/20053/27/ CPST

33 Initial Adult Consumer Production Analysis Create date range for acceptable OCO administrations UCIFirst Svc. Date1st Svc. Minus 601st Svc. Plus /10/20055/11/20058/23/ /25/20055/26/20059/7/2005

34 Initial Adult Consumer Production Analysis Merge Adult Consumer A and Adult Consumer B OCO administrations Match by date when acceptable date exists Determine production ratio/percentage

35 January 2005February 2005March 2005 Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa April 2005May 2005June 2005 Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Period under review (4 th qtr. FY 2005)

36 January 2005February 2005March 2005 Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa April 2005May 2005June 2005 Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa st service in period May 11, 2005

37 January 2005February 2005March 2005 Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa April 2005May 2005June 2005 Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Absence of service determined for 120 day period prior to first service (ensures client is new or in new episode of care)

38 January 2005February 2005March 2005 Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa April 2005May 2005June 2005 Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Outcome administration expected/sought between March 12 and June 24

39 HCCMHB OCO Production Performance Improvement Measures Initial Measures –Adult Consumer

40 HCCMHB OCO Production Performance Improvement Measures Initial Measures –Adult Consumer –Adult Provider –Youth Consumer –Youth Provider –Youth Parent Six-Month Measures –Adult Consumer –Adult Provider –Youth Consumer –Youth Provider –Youth Parent 12-Month Measures –Adult Consumer –Adult Provider –Youth Consumer –Youth Provider –Youth Parent Long-Term Measures –Adult Consumer –Adult Provider –Youth Consumer –Youth Provider –Youth Parent

41 June, 2004 – Contract Language Established Obligating Agencies to 70% Compliance on OCO Administrations

42 July, 2004 – HCCMHB Issues First “Test” Measures

43

44 PI Products Provided to Agencies:

45 UPID Type NumberType Total Expected Submitt ed Not Submitted Ratio expectationRatiocalc Ratio Met Adult Consumer initial yes Adult Provider initial no Child Provider initial yes Child Parent initial yes Child Consumer initial yes Adult Consumer 06 month no Adult Provider 6 month no Child Provider 6 month yes Child Parent 6 month no Child Consumer 6 month no Adult Consumer 12 month no Adult Provider 12 month no Child Provider 12 month no Child Parent 12 month no Child Consumer 12 month505700no Adult Consumer Annual yes Adult Provider Annual yes Child Provider Annual yes Child Parent Annual yes Child Consumer Annual yes

46

47 Spreadsheet containing: UCI First service in period date Last service in period date OCO time period parameters Submission status Date of administration for submission Age of client Service type by numeric rank Instrument type of submission Admission date

48 Agencies may formally dispute findings on a case-specific basis

49 Measures allow agencies to look closely at specific cases missing administrations to discern issues

50 Had to educate agencies on intricacies of logic Tracking based upon most recent intake date

51 Had to cut through many “trivial” issues that impacted few clients but served as a distraction from dealing with larger issues (those effecting many expected administrations)

52 Had to deal with misunderstandings or “loose” understandings of the administration criteria (ex. Use of “person unable to complete” disposition for unplanned termination/discharges)

53 Had to move past focusing upon issues related to the logic used in the PI exercise to focus on agency issues needing attention

54 Had to manage areas subject to manipulation (ex. Use of “B” consumer administrations when “A” is warranted)

55 ….And in the End….. (Measures of Progress Over Time)

56 The Heavens Opened!!!!

57

58 Measures of Progress Relative to Other Board Areas

59

60

61

62

63

64

65

66

67

68

69 Additional “learnings” –120 day absence without new OCO/tx. Plan update – not clinically sound practice –Delayed tx. Following assessment

70 Data Use

71 Ohio Scales Data

72 Ohio Scales State Data Report

73 Youth Consumer Findings

74

75

76 Youth Parent Findings

77

78

79 Youth Worker Findings

80

81

82 Does Hamilton County data look like Statewide data?

83 Youth Consumer Findings

84

85

86 Youth Parent Findings

87

88

89 Youth Worker Findings

90

91

92 Does a longitudinal design change the picture?

93 Youth Consumer Findings

94

95

96 Youth Parent Findings

97

98

99 Youth Worker Findings

100

101

102 Does looking at everyone together make sense?

103 Should the data be apportioned in some manner when considering outcomes? By Diagnosis?

104 HCCMHB expenditures for treatment of youth with diagnosis of (AD/HD) from September 2001 (inception of OCO/OYS) to ~ July, 2005 = $14,470,290 and provided care to 2,652 individuals under the age of 18.

105

106

107

108

109

110

111 Summary: Findings for all groups (Youth, Parents, & Agency Staff) indicate that statistically significant gains are made during the initial period (6 months) of treatment. Findings did not discern statistically significant differences beyond this point.

112 Examination of CY 2001 data indicates that of all children with dx.=314.01, 45% receive greater than 180 days of treatment. 25% receive greater than 375 days of treatment. 11% receive greater than 2 years of treatment.

113 Considerable disparity exists between agencies in regard to the average period of treatment for children with this diagnosis.

114

115 Ohio Consumer Outcomes Data Mart

116 Erik R. Stewart, Ph.D. Vice President, System Performance Renee Kopache, M.S. Recovery Coordinator


Download ppt "Using Outcomes in the Quality Improvement Process: Setting System Level Standards Erik R. Stewart, Ph.D. Vice President, System Performance"

Similar presentations


Ads by Google