1. Quality Indicator Survey (QIS) Design and Development
Martha Powell, PhD
Assistant Research Professor
Division of Health Care Policy and Research
September 13, 2012

2. Presentation Overview
University of Colorado’s role in implementation and analysis of QIS
Development and Implementation of the QIS
Threshold development and review
QIS in Indiana
University of Colorado Anschutz Medical Campus

3. CU’s Role Software design and testing
Stage 1 and Stage 2 revisions/improvements
Desk Audit Reports
Threshold analysis
Help Desk
Onsite support for SA/RO
University of Colorado Anschutz Medical Campus

4. Survey Process Traditional QIS Both shaped by OBRA ‘87
Paper forms, prelim sample from QM/QI, tour used to supplement sample, comp review then focused reviews
QIS
Computer assisted (tablets), random samples, wide variety of indicators, prelim invest then in-depth for selected areas
Both shaped by OBRA ‘87
University of Colorado Anschutz Medical Campus

5. History of QIS
Pre-cursor to QIS (Quality Indicator Survey) developed to serve as standard to gauge effectiveness of survey process improvements
Development involved Univ. of Wisconsin, Univ. of Colorado, and programming contractors
Pilot Tests (field, alpha, beta), then Demonstration and Evaluation
National Implementation—phased approach
University of Colorado Anschutz Medical Campus

6. QIS Basics Computer-assisted (tablets), 2 stages
Stage 1: Preliminary Investigations
Random sampling
Structured protocols
Stage 2: In-depth investigations
Facility tasks (e.g., Kitchen, Infection Control & Immunization Review)
Surveyor Initiation option
Typically four surveyors onsite for 5 days
University of Colorado Anschutz Medical Campus

7. Stage 1 Census Sample (N=40): Admission Sample (N=30):
Resident Interview/Observation
Staff Interview
Record Review
Family Interview (n=3)
Admission Sample (N=30):
recently-admitted residents
MDS Sample: assessments from prior 180 days
University of Colorado Anschutz Medical Campus

8. QCLIs Quality of Care and Quality of Life Indictors
Examples of QCLIs: Underweight and No Supplements, Eating Decline Since Admission, Hospitalization within 30 days of Admission
Computed from Stage 1 and MDS samples/data
Facility rate is calculated and compared to pre-set threshold
If facility rate exceeds threshold, the care area (e.g., Nutrition, ADL, Hospitalization) is triggered for a Stage 2 investigation (e.g., 4 residents out of 40 have Stage III/IV pressure ulcers—10% facility rate—exceeds 5% threshold)
University of Colorado Anschutz Medical Campus

9. Thresholds Rates – threshold = 10% (example rate)
For a census sample of 40, 10% would be 4 residents, so 5 would be needed to exceed threshold
For an admission sample of 30, 10% would be 3 residents, so 4 would be needed to exceed threshold
Threshold for Abuse is 0—only 1 resident is needed to exceed the threshold
Not percentiles or benchmarks
Reviewed and reset if appropriate
Based on QIS data
Separate threshold values for smaller census/admission samples
University of Colorado Anschutz Medical Campus

10. Simplified QCLI Threshold Analysis
Survey
QCLI Rate
Cited?
19% 1
20% N 2 3
22% 4
35% 5
40%
Y  6
41% 7
50%
Y  8
70%
If current threshold is 19%
8 – TRIGGERED and – CITED
3/8 or % citation rate
University of Colorado Anschutz Medical Campus

11. Simplified QCLI Threshold Analysis
Survey
QCLI Rate
Cited?
19% 1
20% N 2 3
22% 4
35% 5
40%
Y  6
41% 7
50%
Y  8
70%
If current threshold is 37%
4 – TRIGGERED and – CITED
3/4 or % citation rate
University of Colorado Anschutz Medical Campus

12. Simplified QCLI Threshold Analysis
Survey
QCLI Rate
Cited?
19% 1
20% N 2 3
22% 4
35% 5
40%
Y  6
41% 7
50%
Y  8
70%
If current threshold is 45%
2 – TRIGGERED and – CITED
2/2 or % citation rate
University of Colorado Anschutz Medical Campus

13. Small/Not Small Sample Thresholds
Census Sample
Small: 35 or fewer residents
Not Small: 36 – 40 residents
Admission Sample
Small: 9 or fewer residents
Not Small: 10 – 30 residents
University of Colorado Anschutz Medical Campus

14. Stage 2
In-depth investigations of triggered care areas and non-mandatory facility tasks
Critical Element Pathways guide investigation
Additional interviews, observations, and record reviews
Compliance decision
Is facility in compliance with regulations?
Individual and then as team
University of Colorado Anschutz Medical Campus

15. QIS Surveyor Training After SMQT
Extensive Classroom training (computer basics, QIS process and software)
Mock survey—putting training into practice
Compliance surveys—performance is evaluated by QIS Trainers
Additional classroom and onsite education to become a QIS Trainer
University of Colorado Anschutz Medical Campus

16. QIS Implementation
University of Colorado Anschutz Medical Campus

17. QIS in Indiana First surveys conducted in January, 2011
As of 7/17/2012—212 QIS ( 500 NH)
15 of 21 teams trained
Full implementation can take up to 3 yrs.
Following slides/results are based upon QIS from 1/28/2011 – 7/17/2012; National and Indiana QIS results from same time period
University of Colorado Anschutz Medical Campus

18. Triggered Care Areas
National and Indiana rates are similar for the following:
Frequently Triggered (> 60% of surveys)
Abuse/Abuse Prohibition, Accidents, ADLs, Community Discharge, Dental Status, Nutrition, Personal Property, Pressure Ulcers, Psychoactive Meds
Less Frequently Triggered (< 20% of surveys)
Food Quality, Notification of Change, Privacy, Sufficient Nursing Staff, Social Services
University of Colorado Anschutz Medical Campus

19. Triggered Care Areas cont’d
Indiana is a low outlier (compared to National QIS rates) for:
Physical Restraints and Positioning
No high outliers for Indiana for triggered care areas
University of Colorado Anschutz Medical Campus

20. Source of “Negative” Responses
Indiana QIS rates for:
Resident Observation 5%
Resident Interview 7%
Family Interview 3%
Staff Interview 10%
Census Clinical Record 5%
Admission Clinical Record 5%
These rates are similar to Nat’l QIS rates.
University of Colorado Anschutz Medical Campus

21. Citation Rates for Mandatory Facility Tasks
Liability Notices 25% (↑than Nat’l QIS)
Dining 23%
Infection Control 46% (↑than Nat’l QIS)
Kitchen 27%
QA & A 13% (↑than Nat’l QIS)
Resident Council 13% (↑than Nat’l QIS)
Med. Admin 13%
Med Storage 22%
Unnecessary Med Use 39%
University of Colorado Anschutz Medical Campus

22. Citation Rates for Non-Mandatory Facility Tasks
Abuse Prohibition 23%
Adm, Tfr, DC 22%
Environment 74%
Personal Funds 26%
Sufficient Nursing Staff 13%
Extended Survey 60%
Rates = # cited divided by # investigated (not total # of surveys)
University of Colorado Anschutz Medical Campus

23. Citation Rates for Triggered CA/Tasks
Indiana QIS similar to Nat’l QIS for most
Indiana QIS low outlier for:
Behavioral and Emotional Status, Rehabilitation, UTIs
Indiana QIS high outlier for:
Physical Restraints and Positioning
University of Colorado Anschutz Medical Campus

24. Scope and Severity A 3.6% 1.2% B 4.8% 2.7% C 3.9% 3.0% D 64.3% 65.3%
Percentage of F-tags at each S/S level for Indiana QIS very similar to Nat’l QIS, slightly higher percentages at A & B
IN QIS Nat’l QIS
A 3.6% 1.2%
B 4.8% 2.7%
C 3.9% 3.0%
D 64.3% 65.3%
E 17.6% 19.9%
F 3.2% 4.8%
G 2.5% 2.4%
H – L 0.2% 0.55%
University of Colorado Anschutz Medical Campus

25. Most Frequently Cited F-tags
Nationally in QIS states
F441 Infection Control %
F329 Drug Regimen free from Unecc. Drugs %
F323 Free of Accident Hazards %
Indiana QIS
F282 Srvcs by Qualified Persons per Care Plan %
F323 Free of Accident Hazards %
F279 Develop Comp. Care Plans %
University of Colorado Anschutz Medical Campus

26. Citation Rates for Selected F-tags
Nat’l QIS IN QIS diff
F % 44.7% -23.6%
(Svcs by Qual. Pers per CP)
F % 28.9% -16.9%
(Notice of Rights)
F % 32.0% -12.5%
(Housekeeping)
University of Colorado Anschutz Medical Campus

27. Citation Rates for Selected F-tags
Nat’l QIS IN QIS diff
F % 22.3% 5.5%
(Right to particp. In CP)
F % 4.1% 5.8%
(Srvcs. meet Professional Stds.)
F % 17.8% 6.5%
(Comprehensive Assessments)
University of Colorado Anschutz Medical Campus

28. Citation Rates for Selected F-tags
Nat’l QIS IN QIS
F % 0%
(Comply w/Fed/St/Local Laws/Prof. Stds.)
F % 0%
(Comp. Assess. After Sig Change)
F % 0%
(Nurse Aide Perform Rev-12 hr/yr in-service)
University of Colorado Anschutz Medical Campus

29. Citations and Survey Time
Nat’l QIS IN QIS Average Number of Citations per Survey Percentage of Citation- Free Surveys 6.9% 9.6% Survey Time (Prep, Onsite, Offsite)
University of Colorado Anschutz Medical Campus

30. Contact Information
University of Colorado QIS website CMS Project Officers Susan Joslin and Tom Kress
University of Colorado Anschutz Medical Campus

31. Questions?
Martha Powell (303) Sayuri Kelly (303) For CMS QIS forms and Information:
University of Colorado Anschutz Medical Campus