Using State-Level Performance Data: an Update on the National CIP Val Bradley and Sarah Taub Human Services Research Institute

How Did We Get Here?  Initial concentration on rudiments of custodial care  Evolution of programmatic standards that reflected growth in our understanding of people’s needs and our own expertise  Increased concern about the efficacy of our interventions  Attention to the outcomes of services and supports from the perspective of the customer

Signs of Change in Performance Management  No longer just better than the institution  Rooted in outcomes  Emphasis on enhancement  Changing role of the state  Changes in experiences and expectations of families and consumers  Changes in accreditation approaches Outcomes Expectations Inclusion

More Signs of Change  Movement away from prescriptive standards  Emphasis on CQI  Collaborative development of standards  Inclusion of customer satisfaction Satisfaction CQI Consensus

Changing Quality Landscape  Exposure of fault-lines in the system (e.g., HCFA and the press)  Expansion of supports to individuals on the waiting list  Emergence of self-determination  Olmstead decision  Struggles with MIS applications  Direct support staff shortages

Emergence of Performance Indicators  First appeared in behavioral and acute care  Provide some “cues” for managing these complex systems  Highlight impact of cost containment  Illuminate what’s working  Provide early warning signs

Characteristics of Performance Indicators  Reflect major organizational or system goals.  Address issues that can be influenced by the organization or system  Have face validity  Point a direction  Reflect rates or major events  Related to associated standards

Consumer Involvement in Assuring Quality  Choice among providers/ purchasing decisions  Designing and refining QA mechanisms  Grievance/appeals process  Consumer satisfaction surveys  Membership on policy-making board  “Consumer Reports”

Continuous Quality Improvement  Leadership at the senior level  Engagement of multiple constituencies  Development of benchmarks  Identification of change strategies  Measurement of progress

Project Beginnings  NASDDDS and HSRI collaboration  Launched in 1997  Seven field test states + steering committee  ~60 candidate performance indicators  Development of data collection instruments

Current Participating States WA IN AZ UT NC IL IA OK VT WV KY PA MA DE RI CT MT WY NE HI Orange County WA IN AZ UT NC IL IA OK VT WV KY PA MA DE RI CT MT WY NE HI Orange County AL SD SC

What will CIP accomplish?  Nationally recognized set of performance and outcome indicators for developmental disabilities service systems  Benchmarks of performance  Trend data at the state & national level  Reliable data collection methods & tools

What are the Core Indicators?  Consumer Outcomes: Satisfaction, choice, employment  Provider Agency/Workforce Stability: Staff turnover  System Performance  Protection of Health and Safety

Data Sources  Consumer Survey  Family Surveys Family Support Survey (adult lives at home) Children/Family Survey (child lives at home) Guardian/Family Survey (adult lives out-of-home)  Provider Survey  DD System MIS (state-level)

Selected Findings 1999 and 2000 Data

Family Survey (2000)

Family/Guardian Survey (2000)

Children/Family Survey (2000)  84.7% of respondents choose the agencies or providers that work with their family some or most of the time  73.1% of families choose the support staff that work directly with their family (some or most of the time)  87.3% of respondents would like at least some control over the hiring and management of their support workers, yet only 67.2% feel they have “some” or more control over this hiring and management

Community Inclusion

Choice and Decision-Making

Consumer employment data (1999) Where people work: Duplicated counts Aggregate N = 3900 (11 states) 27.7% -- supported employment 21.7% -- group employment (enclave/crew) 40.4% -- facility-based employment 36.8% -- non-vocational day supports

Employment by state (1999)

Health & Safety Outcomes Knowing how to file a grievance Feeling safe in your neighborhood Having checkups with doctor and dentist Being free from major/serious injuries Not taking psychotropic medications if you don’t need to Being safe from crime

Health Outcomes (2000)

Consumer Outcomes (2000)  Access 79% of respondents reported that they almost always have a way to get where they want to go  Safety 94% of respondents report feeling safe in their neighborhoods 96% report feeling safe at home

Rate of direct support turnover Average length of time on the job Vacancy rate Staff qualifications and competency considered but postponed; considered a staff survey Staff Stability

Staff Stability (1999)  Day support providers report: Lower turnover Current staff have been employed longer About half as many vacant positions(both FT and PT)  Both types of agencies report: Staff who left within the last year were employed on average about 19 months Part-time position vacancies are much higher than full- time position vacancies

Staff Turnover (1999)  Day Support Agencies 31.2% turnover (n=294) Separated staff employed average of 19.4 months (n=242) Current staff employed average of 40.3 months (n=290)  Residential Support Agencies 35.2% turnover (n=283) Separated staff employed average of 19.5 months (n=259) Current staff employed average of 37.8 months (n=272)

Staff Vacancies (1999)  Day Support Agencies 5.9% of full-time positions are vacant (n=222) 8.7% of part-time positions are vacant (n=167)  Residential Support Agencies 9.9% of full-time positions are vacant (n=217) 18.9% of part-time positions are vacant (n=199)

Representation on Boards (1999)  Across all providers reporting (N = 302) 3.2% of board members are consumers 19.0% of board members are family

Board membership across states (1999) Consumer & Family Representation on Boards

For more information… Visit HSRI’s website:

What Do You Do With the Information?  Include at your web site  Prepare annual reports  Develop provider profiles  Use with sister agencies  Use in allocation decisions  Use to spot red flags

Final Words “ Beware the Continuous Improvement of Things Not Worth Improving ” W. Edward Deming