1. Expanding the Uses of AHRQ’s Prevention Quality Indicators: Validity from the Clinician Perspective Presented by: Sheryl Davies, MA Stanford University Center for Primary Care and Outcomes Research AHRQ Annual Meeting September 26 – 29, 2010 Bethesda, MD

2. Acknowledgements Project team: Sheryl Davies, MA (Stanford) Kathryn McDonald, MM (Stanford) Eric Schmidt, BA (Stanford) Ellen Schultz, MS (Stanford) Olga Saynina, MS (Stanford) Jeffrey Geppert JD (Battelle) Patrick Romano, MS, MD (UC Davis) AHRQ Project Officer: Mamatha Pancholi This project was funded by a contract from the Agency for Healthcare Research and Quality (#290-04-0020)

3. Potentially Avoidable Hospitalizations Admissions for diagnoses that may have been prevented or ameliorated with currently recommended outpatient care Admissions for diagnoses that may have been prevented or ameliorated with currently recommended outpatient care Two independently developed measure sets primarily used in the literature Two independently developed measure sets primarily used in the literature –John Billings –Joel Weissman Strong independent negative correlations between self- rated access and avoidable hospitalization Strong independent negative correlations between self- rated access and avoidable hospitalization Correlations between avoidable hospitalization and: Correlations between avoidable hospitalization and: –household income at zip code level (neg) –uninsured or Medicaid enrolled (pos) –maternal education (neg) –physician to population ratio (neg) –Weaker associations for Medicare populations

4. Prevention Quality Indicators Background Developed in early 2000s Developed in early 2000s Numerator: Number of admissions within a geographic area Numerator: Number of admissions within a geographic area Denominator: Population Denominator: Population Some admissions are excluded if considered relatively less preventable Some admissions are excluded if considered relatively less preventable Conditions selected had adequate variation, signal ratio, and literature based evidence supporting use Conditions selected had adequate variation, signal ratio, and literature based evidence supporting use

5. Prevention Quality Indicators Diabetes related indicators Diabetes related indicators –Diabetes, short-term complications (PQI 1) –Diabetes, long-term complications (PQI 3) –Lower extremity amputations among patients with diabetes (PQI 16) Chronic disease indicators Chronic disease indicators –Chronic obstructive pulmonary disease (PQI 5) –Hypertension (PQI 7) –Congestive heart failure (PQI 8) –Angina without procedure (PQI 13) –Adult asthma (PQI 15) Acute disease indicators Acute disease indicators –Perforated appendicitis (PQI 2) –Dehydration (PQI 10) –Bacterial pneumonia (PQI 11) –Urinary infections (PQI 12)

6. Potential uses of PQIs QI Comp Report P4P AreaX PayorXX ProviderXXX LTCXXX 1 We initially assessed the internal quality improvement application for large provider groups. Following our initial rating period, panelists expressed interest in applying select indicators to the long term care setting and these applications were added to our panel questionnaire. Current application Extended applications Extended application proposed by panel

7. Scenarios of use Area level – Publish maps of rates by county. Target areas with higher rates Area level – Publish maps of rates by county. Target areas with higher rates Payors (SCHIP, Medicare Advantage, private plans) Payors (SCHIP, Medicare Advantage, private plans) –CR: Publicly report payor rates to improve consumer choice –P4P: Medicaid agencies implementing P4P for contracted payor groups Provider (large provider groups)/LTC Provider (large provider groups)/LTC –QI: Analyze rates to identify potential intervention targets (e.g. care coordination, education) –CR: Publicly report provider rates to improve consumer choice –P4P: Payors implementing P4P programs for contracted provider groups

8. Methods Clinical Panel review using new hybrid Delphi/Nominal Group technique Clinical Panel review using new hybrid Delphi/Nominal Group technique Two groups: Core and Specialist Two groups: Core and Specialist –Core assesses all; Specialist only applicable Three indicator groups: Acute, Chronic, Diabetes Three indicator groups: Acute, Chronic, Diabetes Two panels: Two panels: –Delphi –Nominal Group

9. DelphiDelphi rating Results: initial rating Delphi comments Nominal comment NominalNominal rating Results: Initial rating 1 st round results to panelists prior to call Diabetes call Acute call Chronic call Nominal panel re-rates Call summaries to panels Final ratings Delphi panel re-rates Panel Process: Exchange of Information

10. Quality Improvement Applications Indicator Provider (Delphi/Nominal) COPD and Asthma (40 yrs +) ▲▲▲▲▲ Asthma ( < 39 yrs) ▲▲▲▲▲▲ Hypertension▲▲▲▲▲ Angina▲▲▲▲ CHF▲▲▲▲▲▲ Perforated Appendix ▲▲▲ Diabetes Short Term Complications ▲▲▲▲▲▲ Diabetes Long-Term Complications ▲▲▲▲▲ Lower Extremity Amputation ▲▲▲▲▲ Bacterial Pneumonia ▲▲▲▲ UTI▲▲▲▲ Dehydration▲▲▲ ▲ Major Concern Regarding Use, ▲▲ Some Concern, ▲▲▲ * Majority Support, ▲▲▲ Full Support

11. Comparative Reporting Applications IndicatorAreaPayorProvider COPD ▲▲ / ▲▲ ▲▲ / ▲▲▲ Asthma ( < 39 yrs) ▲▲ / ▲▲▲ Hypertension ▲▲ / ▲▲ Angina ▲ / ▲ CHF ▲▲ / ▲▲▲ ▲▲▲ / ▲▲▲ Perforated Appendix ▲▲ / ▲ Diabetes Short Term ▲▲ / ▲▲ ▲▲ / ▲▲▲ Diabetes Long-Term ▲▲ / ▲▲▲ ▲▲ / ▲▲ LE Amputation ▲▲▲ / ▲▲▲ ▲▲ / ▲▲▲ ▲▲ / ▲▲ Bacterial Pneumonia ▲▲ / ▲▲ UTI Dehydration ▲▲ / ▲ ▲ / ▲ ▲ Major Concern Regarding Use, ▲▲ Some Concern, ▲▲▲ * Majority Support, ▲▲▲ Full Support

12. Pay for Performance Applications IndicatorPayorProvider COPD ▲▲ / ▲▲ ▲▲ / ▲▲▲ Asthma ( < 39 yrs) ▲▲ / ▲▲ ▲▲ / ▲▲▲ Hypertension ▲▲ / ▲▲▲* ▲▲ / ▲▲ Angina ▲▲ / ▲ CHF ▲▲ / ▲▲ Perforated Appendix ▲▲ / ▲ Diabetes Short Term ▲▲ / ▲▲ Diabetes Long-Term ▲▲ / ▲▲ Lower Extremity Amputation ▲▲ / ▲▲ Bacterial Pneumonia ▲▲ / ▲▲ UTI ▲▲ / ▲ Dehydration ▲ / ▲ ▲ Major Concern Regarding Use, ▲▲ Some Concern, ▲▲▲ * Majority Support, ▲▲▲ Full Support

13. Concordance Between Panels Delphi Full supportDelphi Some Concern Delphi Major Concern NG Full support821 (6) 1 0 NG Some concern0340 NG Major Concern012 (5) 1 3 1 Numbers in parentheses are the number of instances in that cell where │Median (Delphi) – Median (NG)│> 1.  Majority of combinations rated the same (56%).  Three combinations had one rating of “majority support” which requires disagreement within one panel (not shown on table).  Of remaining differences, all were within one level. Of those about 2/3 had a difference in medians of one or less.  Delphi panel always more moderate than NG

14. What feeds into the ratings?

15. Delphi vs. Nominal Delphi group Delphi group –Advantages: Better reliability, more points of view, less chance for one panelist to pull the group –Disadvantage: Less communication and cross-pollination across panelists, less ability to discuss and refine details of indicators/evaluation Nominal group Nominal group –Advantages: Can discuss details, facilitate sharing of ideas –Disadvantages: Limited in size and therefore representation, one strong panelist can flavor group and therefore poorer reliability Linear regression on usefulness ratings Linear regression on usefulness ratings –Mixed model: panelist random effect (nested) –Fixed effects:  Delphi vs. NG (N.S.)  Generalist vs. Specialist (F=32.3, p<.0001)  Public Health vs. Other (F=20.0, p<.0001)  Quality vs. Other (F=54.7, p<.0001)  Denominator Level (F=24.4, p<.0001)  Use (F=23.2, p<.0001)  Indicator (F=8.5, p<.0001)

16. Potential interventions to reduce hospitalizations AcuteChronic Area  Access to primary care/urgent care  Access to care  Lifestyle modifications Payor  Coverage of medications  Coverage of auxiliary health services (e.g. at home nursing)  Access to primary care/urgent care  Coverage of medications  Coverage of comprehensive care programs  Coverage of auxiliary health services (e.g. at home nursing)  Disease management programs  Lifestyle modification incentives Provider  Quality nursing triage  Patient education  Accurate/rapid diagnosis and treatment  Appointment availability  Outpatient treatment of complications  Education, disease management  Lifestyle medication interventions  Comprehensive care programs, care coordination, auxiliary health services

17. So you want to adapt the PQI? Selecting indicators Selecting indicators –Stability of denominator group improves validity for long-term complications Defining the numerator Defining the numerator –One admission per patient per year –Using related principal dx with target secondary dx –Including first hospitalization before chronic condition dxed Defining the denominator Defining the denominator –Identifying patients with chronic diseases (mulitple dx, population rates, pharmaceutical data) –Requiring minimum tenure with payor or provider

18. Risk adjustment Demographics Demographics –Age and gender highly rated as important –Race depending on indicator Disease severity Disease severity –Historical vs. current data Comorbidity Comorbidity –Highly rated as important Lifestyle associated risk and compliance Lifestyle associated risk and compliance –Smoking, obesity –Pharmacy records –Can interventions help reduce impact of these factors? Socioeconomic status Socioeconomic status –Highly rated as important –May mask true disparities in access to care –Panel felt benefits of inclusion outweighed problems

19. Policy implications Ensuring true quality improvement Ensuring true quality improvement –Case mix shifting, coding Cost/burden of data collection Cost/burden of data collection Does avoiding hospitalization really reflect the best Does avoiding hospitalization really reflect the best –Quality –Value

20. Next steps Understanding stakeholder perspectives Results represent clinical perspective Results represent clinical perspective Other stakeholders may be more attuned to public health, access to care, quality uses Other stakeholders may be more attuned to public health, access to care, quality uses Other important perspectives: Other important perspectives: –Public health –Long term Care –Policy-makers –Quality stakeholders Why are there differences in perspectives? Why are there differences in perspectives?

21. Next steps Investigate multiple definitions Investigate multiple definitions Investigate risk adjustment approaches Investigate risk adjustment approaches Continue to learn from user experience Continue to learn from user experience Identify interventions and link usefulness of indicators with true quality improvement Identify interventions and link usefulness of indicators with true quality improvement