Comparing Apples to Apples: Use of Common Tools to Rebalance Systems National HCBS Waiver Conference October 28, 2003 Val Bradley & Sarah Taub Human Services Research Institute

Changing Quality Landscape Exposure of fault-lines in the system (e.g., CMS and the press) Emergence of self-determination Olmstead decision and proposed closures Struggles with MIS applications Direct support staff shortages Expansion of supports to individuals on the waiting list

Signs of Change in Performance Management No longer just better than the institution Rooted in outcomes Emphasis on enhancement and CQI Changing role of the state Changes in experiences of families and people with mental retardation Changes in accreditation approaches

More Signs of Change Movement away from prescriptive standards to individualized risk management Collaborative development of standards Inclusion of consumer and family participation in oversight Satisfaction CQI Consensus

Emergence of Performance Indicators First appeared in behavioral and acute care Provide some “cues” for managing these complex systems Highlight impact of cost containment Illuminate what’s working Provide early warning signs

Characteristics of Performance Indicators Reflect major organizational or system goals. Address issues that can be influenced by the organization or system Have face validity Point a direction Reflect rates or major events Related to associated standards

NCI Beginnings NASDDDS and HSRI collaboration Launched in 1997 Seven field test states (plus steering committee) ~60 candidate performance indicators Development of data collection protocols

What has NCI Accomplished? Nationally recognized set of performance and outcome indicators for developmental disabilities service systems Reliable data collection methods & tools (consumer & family surveys, provider survey, system data) Baseline and trend data at the state & national level

Participating NCI States ND AK HI WA OR ID MT WY ND SD NE CA UT NV AZ CO NM KS OK TX MN IA WI IL IN MO AR MI OH KY TN MS LA AL GA FL SC NC VA WV PA NY ME VT NH MD DC DE NJ CT MA RI PR Orange County

NCI Structure Currently 20 states plus Orange County in Phase VI (FY2004) HSRI provides technical assistance under subcontract to NASDDDS Subcommittees address specific issues Meet with full steering committee annually

Where does NCI fit in? One component of state Quality Management systems In many participating states, main process for measuring consumer and family satisfaction Integration of information is the key (many QA systems are fragmented) Increasing interest in using NCI to measure provider performance

What are the data sources? Consumer Survey Face-to-face interview Minimum n=400 Standardized training provided High inter-rater reliability First section (subjective) allows consumer responses ONLY Provider Survey Staff Stability Board Representation Family Surveys Mail surveys (goal=40% return) Adult Family Survey (at home, 18+) Family Guardian Survey (out-of-home) Children Family Survey (at home, <18) System Data Incidents Mortality

How is NCI data used? Setting goals and strategic planning Setting priorities for quality improvement, Quality Management plans Budget requests to Governor and legislators Helps shape data reporting systems (e.g., incidents, mortality) Stakeholder advisory committees (Quality Councils)

More specific uses… Pennsylvania – part of Independent Monitoring and quality improvement South Carolina – Core component of external monitoring Wyoming – Annual reports, CMS review* Massachusetts – Strategic planning North Carolina – Health status indicators *See Wyoming presentation: “Using CORE INDICATORS in Federal HCBS Reviews”

How are results disseminated? Reports of state results vs. national results posted on websites Presentations to staff, providers, community Summaries shared with families who filled out surveys (AZ) Simplified version of Consumer Survey report for self-advocates (VT)

Trend Analysis Beginning to look at trends over past three years Five states collected Consumer Survey data annually for three years Connecticut Kentucky North Carolina Pennsylvania Rhode Island

CM helps get what person needs… 90% 88% 83% 78% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 5 state averageAll state average FY00FY01FY02

92.8% 93.6% 94.4% 92% 93% 94% 95% FY00FY01FY02 Person is satisfied with home…

0% 20% 40% 60% FY00FY01FY02 Housemate Home staff Case manager Person had input in choosing…

Staff Stability 35.2% 41.7% 52.9% 35.8% 45.1% 31.2% 0% 10% 20% 30% 40% 50% 60% residential day

Next steps… Expanding indicators on self-directed services and supports Adapting survey for people with different communication styles Adding health status indicators based on NC supplement Creating templates for displaying information Continuing trend analysis

For More Information Reports are available on HSRI’s website: Contact: