1. IUMSP Institut universitaire de médecine sociale et préventive, Lausanne 1 Jean-Marie Januel, PhD, MPH, RN Senior Researcher University hospital of Lausanne, Institute of Social and Preventive Medicine, Lausanne, Switzerland Health Services Research Seminar Series UC Davis, Center for Healthcare Policy and Research Sacramento, CA, US – May 9, 2013

2. Disclosure Statement Speaker’s Verbal Disclosure Statement: Have you (or your spouse/partner) had a personal financial relationship in the last 12 months with the manufacturer of the products or services that will be discussed in this CME activity? ___ Yes _X_ No (If yes, please state disclosures and resolutions)

3. Educational objectives  To describe the French experience of PSI (focusing on feasibility, validity and measurement variations between providers)  To describe the Swiss experience of PSI (focusing on the use of PSI for assessing the impact of DRG implementation in all Swiss hospitals)  To discuss about an appropriate reference standard for comparing Patient Safety Indicators (example provided by a pilot study to compare the PSI 12 – postoperative pulmonary embolism and deep vein thrombosis – in hospitalized patients undergoing hip arthroplasty between France and Switzerland)

4. BACKGROUND

5. Hypothesis to Develop New Indicators  To use Administrative Databases for assessing adverse events related to healthcare that could be preventable McDonald K, Romano P, Geppert J, Davies SM, Duncan BW, Shojania KG, et al. Measures of Patient Safety Based on Hospital Administrative Data - The Patient Safety Indicators. Technical Review 5. AHRQ Publication No. 02-0038. Rockville, MD: Agency for Healthcare Research and Quality  Approach derived from the Complication Screening Program (CSP) by Iezzoni LI et al. Iezzoni LI, Foley SM, Heeren T, Daley J, Duncan CC, Fisher ES, Hughes J. A method for screening the quality of hospital care using administrative data: preliminary validation results. QRB Qual Rev Bull. 1992;18(11):361-371.

6. ICD Codes for Secondary Diagnoses related to Adverse Event Identification Population at risk, defined using DRG codes, Diagnostic codes, Procedure codes PSI = PSI Algorithm Model

7. Projects  20 Patient Safety Indicators (PSI) initially developed by the Agency for Healthcare Research and Quality (AHRQ) using ICD9-CM McDonald K, et al. Measures of Patient Safety Based on Hospital Administrative Data - The Patient Safety Indicators. Technical Review 5. AHRQ Publication No. 02-0038. Rockville, MD: Agency for Healthcare Research and Quality  ICD10 adaptation of 15 PSI by the International Methodology Consortium for Coded Health Information in 2007 (www.imecchi.org)www.imecchi.org Quan H., et al. AHRQ Advances in Patient safety 2: News Directions and Alternatives Approaches. Vol 1. Assessment. Taxonomies and Measurement. Agency for Health Care Research and Quality Publication 2008. Januel JM, et al. Rev Epidemiol Sante Publique 2011;59 :341-350.

8. Achievements for PSI  Benchmark (external comparisons) o Between healthcare providers (hospitals) o Between healthcare systems (countries)  Surveillance (internal comparisons) o Longitudinal studies (to assess epidemiological peak) o To assess changes in robustness relating of changes in ICD coding rules / version used

9. The Patient Safety Indicators in the Literature *Including one citation without abstract; **including two citations without abstract.

10. The French experience of PSI

11. French Pilot Study – Objectives  To explore French data using PSI algorithms (ICD10 version - IMECCHI)  To assess feasibility for PSI by calculating a selection of PSI from nationwide database in 2005 and 2006, respectively  To validate selected PSI using chart review

12. PSI estimates calculation for feasibility

13. Stratified analysis for PSI #12 (Postoperative PE /DVT) Men Women Men

14. Estimating PPV based on three PSI  To compare PSI positive cases to a retrospective chart review of medical records  Between 80 and 150 inpatients stays with positive PSI at the university hospital of Lyon, France  To calculate the Positive Predictive Value (PPV)  To identify potential reasons for false positive cases

15. PPV of PSI #12 (postoperative PE/DVT) University Hospital, Lyon, France (2008)

16. PPV of PSI #7 (Bacteremia-related to Catheter) University Hospital, Lyon, France (2008)

17. PPV of PSI #13 (postoperative sepsis) University Hospital, Lyon, France (2008)

18. The Swiss experience of PSI

19. IDoC Project (Switzerland) Impact of Diagnosis related groups (DRG) implementation on patient Care and professional practice  To assess the Impact of Diagnosis related groups (DRG) implementation on patient Care and professional practice in Swiss hospitals o Overall Hospitals DRG implementation in Switzerland for inpatient stays payment on 2012, January 1 st  Design o 5 sub-projects based on several outcomes (Ethic; Law; Nursing sensitive; AMI; and PSI)  Sub-project “PSI” o To monitor the possible effects of the generalization of DRG- based hospital reimbursement using Patient Safety Indicators (PSI)

20. To take into account the number of SDx in Models (hierarchical) better  At inpatient Level o To control differences in case mix (using categorical variable)  At Hospital Level o To control differences in quality of coding o By assessing adjusted average number of SDx for each hospitals using negative binomial regression models

21. SDx in a Previous Analysis of PSI #12

22. Impact of Changes in ICD10 version on PSI 12 estimates  Changes in ICD10 version occurred between 2008 and 2010 in Switzerland o 2008: ICD10-WHO o 2009: ICD10-GM2008 o 2010: ICD10-GM2010  Trend comparisons across years (2008, 2009 and 2010) using ICD-10-WHO  To compare ICD-10-WHO to ICD-10-GM using: o Data from 2009 o Data from 2010

23. Hierarchical Logistic Regression Model for PSI #12

24. To Develop a New “PSI 13”  To refine definition from Postoperative Sepsis to Postoperative Hospital-Acquired Infections  To develop a New Algorithm  To compare New Algorithm to PSI 13 (AHRQ)  To repeat models using data from 2008 to 2010, with respect to changes in ICD-10 version used each year in Switzerland

25. Approach for New Algorithm (ICD10)

26. New Algorithm vs. PSI #13 (AHRQ)

27. PSI rate “high” or “low ”? A pilot study to compare Swiss and French data for PSI #12 (Postoperative PE/DVT)

28. To which Benchmark / Reference Standard Compare PSI 12 (postoperative PE/DVT)?  The Hypothesis o Need a BENCHMARK for assessing comparisons across clusters (e.g., hospitals, countries…) in studies for healthcare quality improvement  Possible benchmarks / reference standards o Risk-adjusted observed vs. expected rate (Funnel plots) o Benchmark vs. « best in class » (Forest plots) o Zero event o Evidence-based data / information

29. Development of an evidence-based benchmark for PSI 12 (postoperative PE/DVT)  Proposal: to compare actual values to an evidence-based reference standard  Three step methodology o To develop a “reference standard” using the baseline risk of Postoperative PE/DVT occurring in hospitalized patients undergoing hip arthroplasty, under appropriate prophylaxis, using a systematic review with meta-analysis o To estimate adjusted occurrence rates of PSI 12 in patients undergoing hip arthroplasty (pilot study using data from Switzerland and France) o To compare these rates against the “reference standard” as a benchmark (or a target) developed using the meta-analysis

30. Systematic Review (Januel JM, et al. JAMA 2012;307(3): 294-303)  Original RCT and observational studies published from 1996 to 2011 (PubMed-Medline, EMBASE, Cochrane)  Adult patients undergoing hip arthroplasty with appropriate VTE prophylaxis (updated guidelines during the corresponding period)  Proportions of symptomatic VTE occurring between arthoplasty and hospital discharge using pooled occurrence rate with random effects  GRADE 1 method for assessing quality of included studies and for evaluating the evidence-basis of our systematic review results (1) GRADE working group. www.gradeworkinggroup.org

31. Systematic Review – RESULTS  27 studies  21’369 adult patients undergoing hip arthroplasty  38 prophylactic treatment subgroups  58 to 70 years old on average  Follow-up after surgery ranged from 8 to 17 days  Pooled VTE 0.53% (95% CI, 0.35% to 0.70%) Heterogeneity: I 2 =49.4% P<0.001

32. Pooled estimates by type of prophylaxis

33. PSI 12 estimates using Swiss / French Data  Routine Data (ICD10) from overall Swiss / French hospitals (3 consecutives years)  Hierarchical Two-Level Logistic Regression Model for estimating PSI #12 outcomes o Level-1 = Inpatients o Level-2 = Hospitals

34. Risk-adjustment Model  Fixed effect variables at each level of the model: o Inpatients  Case mix (Sex, Age, Death, # of Secondary Diagnoses coded, Selection of comorbidities from Charlson and Elixhauser indices)  Usual practice for detecting DVT (e.g., lower extremities ultrasound before hospital discharge) o Hospitals  Average number of secondary diagnoses coded (adjusted on sex, age and length of stay using negative binomial regression models)  Number of hip arthroplasty procedures

35. Adjusted PSI 12 using Two-Level Logistic Regression Models

36. Factors associated to VTE  Switzerland o Some comorbidities are different across years (lower frequency of some comorbidities, thus not included in models)  France o The systematic use of lower extremity ultrasound for screening DVT before discharge in almost 20% of inpatients

37. Comparisons of Swiss and France (PSI 12) against reference standard (meta-analysis)  Stratified on Length of Stay  Displayed on a same figure including: o Adjusted proportions of VTE using the PSI #12 (data from Switzerland and France) o Pooled proportion of VTE occurring before hospital discharge, using meta- analysis (LMWH prophylaxis according to ENDORSE Study)

38. LMWH = 0.58% (0.35% - 0.81%) Comparisons stratified on length of stay 76.4%74.1% FranceSwitzerland

39. International Perspectives  Enlargement of the Swiss-French Pilot Study to several other country (Canada, Germany, Australia, New-Zealand, South-Korea, USA …)  Expected issues o Overall nationwide data or 10% (or more) representative sample? o Which countries do not have sufficient data quality for assessing practice using procedure codes? o Which other potential reasons for explaining differences between countries

40. Conclusions

41. Facts …  We are able to compare all outcomes that could be measured for HSR…  But we are only able to interpret measures using the same definition / the same metrology / the same data quality…  Or, for which we are able to control potential biases…

42. A Questionnaire for Conducting Studies to Assess Quality of Healthcare  Which variations across and between providers (hospitals, health systems as countries)?  What are the sources of these variations (Clinical definition, Algorithm codes and Coding rules, Data quality, Case-mix, Quality of care, Random effects)?  How to control these potential biases (models, risk-adjustment variables, interpretation)?

43. Back to Basic…  Conditions for assessing healthcare quality using patient safety indicators based on the relation between a Process, an Outcome, and a Structure o Before hospital discharge (AEs related to healthcare in hospitals / potentially no sufficient consistency and accuracy for data after discharge) o Outcome compared to “state of the art” practices (guidelines, recommendations, systematic review with meta-analysis) o Results should be interpretable for decision making

44. Approach for Modeling Comparisons between Providers / Health Systems  Take into account (adjustment, stratification) o Difference in practice (use of prophylaxis to prevent VTE, see ENDORSE Study) o Difference in measurement (use of different methods for assessing a diagnosis) o Difference in coding diagnoses (Coding rule for Major Diagnosis, # of Secondary Diagnoses coded) o Difference between Health Systems and Hospitalization habits (average length of stay between countries)

45. Giving Sense to our Findings "We have to assess the quality of our measures and make sure that they make sense to clinicians, make sure providers can act upon them and that we account for variation in how sick the patients are." Prof. Patrick S. Romano From CHPR Website

46. IUMSP Institut universitaire de médecine sociale et préventive, Lausanne 46 Jean-Marie.Januel@chuv.ch

47. Inpatients Characteristics

48. Adjusted Average # of SDx at hospital Level using Negative Binomial Regression