1 Measuring Quality Using Experience Surveys 2005 Center for Policy and Innovation Quality Assurance and Improvement January 26, 2006

2 Critical optimism “is the belief that the specific conditions which exist at one moment, be they comparatively bad or comparatively good, in any event may be bettered”. (John Dewey, 1929)

3 DADS Mission T o provide a comprehensive array of aging and disability services, supports, and opportunities that are easily accessed in local communities. Our key responsibilities to the citizens of Texas include:  Working in partnership with consumer, caregivers, service providers, and other stakeholders.  Developing and improving service options that are responsive to individual needs and preferences  Ensuring and protecting self-determination, consumer rights, and safety.

4 Purpose  Obtain information from the person’s perspective about their experiences  Obtain initial baseline of data to build upon  Conduct surveys annually in order to trend and analyze  Data will point to areas for further exploration

5 Measuring Quality  One of only a few states to undertake a project of this size and scope  1,980 face-to-face surveys in the HCS, TxHmL, ICF/MR, CLASS, DB-MD, CBA, and CWP for 18 years and older

6 Introduction  DADS joined National Core Indicator Project in September 2005  Contracted with NACES Plus Foundation, Inc. to conduct the surveys  Interviews conducted May through August  Interviews conducted annually to identify trends

7 Quality Indicators  National Core Indicators (NCI)  30 Indicators  4 Domains – Consumer Outcomes – System Performance – Health, Welfare, and Rights – Self-Determination (added by the QA/QI Task Force)  Participant Experience Survey (PES) Elderly/Disabled version  39 Indicators  5 Domains – Access to Care – Choice and Control – Respect/Dignity – Community Integration/Inclusion – Self-Determination (added by QA/QI Task Force)

8 Methodology  Proportional sampling based on number of people in each program by county  Interviews were conducted in the person’s home unless person chose not to  Interviews were conducted in about one-half of the counties across Texas  Demographics from DADS automated systems of people receiving services

9 Training  Conducted in April 2005  In person by developers of tools  Mock interviews with persons receiving services  Regular conference call with interviewers  Frequently asked questions collected and distributed

10 It is important to use the data as a basis for further discussion and inquiry in order to “interpret” or draw conclusions from the results.

11 Next Steps  Internal review of results  Share with QA/QI Task Force to: – Review results and identify/prioritize areas for improvement or further study – Develop strategies for making improvements  Broader stakeholder input

12 High-Level Results  Majority of people in HCS, TxHmL, ICF/MR, CLASS, DB-MD, and CWP report: – They receive the services they need – Have had a physical exam in the past year – Do not feel lonely

13 Results - continued  Majority of people in HCS, TxHmL, and CLASS are satisfied with where they live  Majority of people in HCS, TxHmL, ICF/MR, CLASS, and DB-MD participate in integrated activities in their communities

14 Results - continued  In the HCS, TxHmL, ICF/MR, CLASS, DB- MD, & CWP a low proportion of: – Women had OB/GYN exams in the last year – People have participated in activities of self- advocacy groups – People were provided options about where to live and work – People made choices about important life decisions including: housing, roommates, support staff or providers, daily routines, social activities, job, and services coordination

15 Results - continued  The majority of people in the CWP & CBA Programs: – Are able to eat when they need to because there is someone there to help them. – Have chosen their current job – Like their current job

16 Results -continued  69% of people in CWP and 39% of people in CBA do not choose their staff but would like to  50% of people in CWP and 64% of people in CBA report an unmet need for community involvement  Over one-third of people in CWP who are not elderly would like to work

17 Home and Community-Based Services (HCS) Highest satisfaction with the following:  93% - “needed” services were available  92% - have had a physical exam in the past year

18 Home and Community-Based Services (HCS) Lowest satisfaction with the following:  16% - people have participated in activities of self-advocacy groups or other groups that address rights  17% - people make choices about important life decisions including: housing, roommates, support staff or providers, daily routines, social activities, job and services coordination

19 Texas Home Living (TxHmL) Highest satisfaction with the following:  93% - “needed” services were available  93% - do not feel lonely

20 Texas Home Living (TxHmL) Lowest satisfaction with the following:  14% - women have had an OB/GYN exam in the past year  9% - people have participated in activities of self-advocacy groups or other groups that address rights  7% - people report having been provided options about where to live  16% - people indicate that most staff who come to their home treat them with respect * *72% reported having no staff who come to their home

21 Intermediate Care Facilities for Persons with Mental Retardation (ICF/MR) Highest satisfaction with the following:  98% - “needed” services were available  96% - have had a physical exam in the past year

22 Intermediate Care Facilities for Persons with Mental Retardation (ICF/MR) Lowest satisfaction with the following:  12% - have participated in activities of self- advocacy groups or other groups that address rights  16% - report having been provided options about where to live  15% - report having been provided options about where to work  9% - people make choices about important life decisions about; housing, roommates,support staff or providers, daily routines, social activities, job, and services coordination

23 Community Living Assistance and Support Services (CLASS) Highest satisfaction with the following:  93% - “needed” services were available  96% - have had a physical exam in the past year

24 Community Living Assistance and Support Services (CLASS) Lowest satisfaction with the following:  10% - women have had an OB/GYN exam in the past year  14% - people report having been provided options about where to live  9% - people report having been provided options about where to work

25 Deaf-Blind with Multiple Disabilities (DB-MD) Highest satisfaction with the following:  98% - “needed” services were available  90% - have had a physical exam in the past year  92% - do not feel lonely Results are valid plus or minus 9.4 percent because of the small sample size.

26 Deaf-Blind with Multiple Disabilities (DB-MD) Lowest satisfaction with the following:  15% - people have participated in activities of self-advocacy groups or other groups that address rights  10% - people report having been provided options about where to live  10% - people report having been provided options about where to work  15% - people have an “advocate” or someone who speaks on their behalf

27 Deaf-Blind with Multiple Disabilities (DB-MD) Lowest satisfaction with the following:  18% - people report that service coordinators help them get what they need  20% - people who report that they feel safe in their home  20% - people who report that they feel safe in their neighborhood  20% - people whose basic rights are respected by others

28 Consolidated Waiver Program (CWP) Using the NCI Survey Highest satisfaction with the following:  88% - “needed” services were available  84% - do not feel lonely  78% - have had a physical exam in the past year Results are valid no more than plus or minus 9.6 percent because of the small sample size.

29 Consolidated Waiver Program (CWP) Using the NCI Survey Lowest satisfaction with the following:  4% - women have had an OB/GYN exam in the past year  20% - people have participated in activities of self-advocacy groups or other groups that address rights  6% - people report having been provided options about where to live  18% - people make choices about important life decisions including: housing, roommates, support staff or providers, daily routines, social activities, job, and services coordination

30 Consolidated Waiver Program (CWP) Using the PES Survey Major findings from the PES indicators:  3% - are not treated respectfully by staff in their homes  3% - are verbally abused by current staff Results are valid plus or minus 4.6 percent because of the small sample size.

31 Consolidated Waiver Program (CWP) Using the PES Survey Other major findings:  69% - do not choose their care staff, but would like to  50% - report an unmet need for community involvement Results are valid plus or minus 12.5 percent because of the small sample size.

32 Community Based Alternatives (CBA) Using the PES Survey Major findings from the PES indicators:  3% - sometimes go without a meal because there is no one there to help them  3% - are unsure who to report staffing problems to or would report staffing problems to “no one”  2% - are sometimes unable to eat because there is no one there to help them

33 Community Based Alternatives (CBA) Using the PES Survey Other major findings:  37% - earn enough money to buy the things they want  39% - do not choose their care staff, but would like to  64% - report an unmet need for community involvement