1. Ryan Mutter, Ph.D. Agency for Healthcare Research and Quality Rockville, MD Michael Rosko, Ph.D. School of Business Administration Widener University Chester, PA September 14, 2009 Using Stochastic Frontier Analysis to Measure Hospital Inefficiency

2. Recent interest in estimating inefficiency arises out of concerns about excessive expenditures in health care. Recent interest in estimating inefficiency arises out of concerns about excessive expenditures in health care. Inefficiency measurement also adds perspective to quality measurement and highlights trade-offs in quality improvement. Inefficiency measurement also adds perspective to quality measurement and highlights trade-offs in quality improvement. – Quality improvement can increase costs by reducing overuse and expensive medical errors. – But quality improvement can also result in higher levels of resource use and higher costs. There are many potential applications for accurate provider-level estimates of inefficiency (e.g., organizational improvement, public reporting). There are many potential applications for accurate provider-level estimates of inefficiency (e.g., organizational improvement, public reporting). Inefficiency Estimation

3. Multi-dimensional Approaches Non-parametricParametricParametricOneDimensionalApproaches Frontier Approaches Average Approaches Performanceindicators StochasticFrontierAnalysis(SFA) CorrectedOrdinaryLeast Squares Squares(COLS) DataEnvelopmentAnalysis(DEA) OrdinaryLeast (OLS) Approaches to Inefficiency Estimation

4. Econometric technique Econometric technique – Generates provider-level (i.e., hospital-level) estimates of inefficiency – Inefficiency estimates are measured as departures from a statistically derived, theoretical best-practice frontier that takes input prices, outputs, product mix, quality, case mix, and market forces into account Stochastic Frontier Analysis (SFA) Stochastic Frontier Analysis (SFA)

5. Total Expenses Output SFA Frontier Inefficiency Random Error SFA (continued)

6. Measures cost inefficiency (i.e., the percentage by which observed costs exceed minimum costs predicted for a given level of outputs, input prices, etc.) Measures cost inefficiency (i.e., the percentage by which observed costs exceed minimum costs predicted for a given level of outputs, input prices, etc.) Particularly useful for determining the relative performance of hospitals Particularly useful for determining the relative performance of hospitals – Hospital A is among the top 40 percent most efficient hospitals in its peer group. Folland and Hofler (2001) demonstrate its usefulness for comparing the efficiency of groups of hospitals Folland and Hofler (2001) demonstrate its usefulness for comparing the efficiency of groups of hospitals

7. SFA (continued) Specified generally as Specified generally as TC i = f(Y i, W i ) + e i where TC represents total costs; Y is a vector of outputs; W is a vector of input prices; and e is the error term, which can be decomposed as follows where TC represents total costs; Y is a vector of outputs; W is a vector of input prices; and e is the error term, which can be decomposed as follows e i = v i + u i where v is statistical noise ~ N(0, σ 2 ) and u consists of positive departures from the cost- frontier where v is statistical noise ~ N(0, σ 2 ) and u consists of positive departures from the cost- frontier

8. Byproduct of the analysis is information about hospital-level variables on cost and environmental pressure variables on inefficiency Byproduct of the analysis is information about hospital-level variables on cost and environmental pressure variables on inefficiency SFA (continued)

9. Data Sources American Hospital Association (AHA) Annual Survey of Hospitals American Hospital Association (AHA) Annual Survey of Hospitals Medicare Cost Reports Medicare Cost Reports AHRQ Healthcare Cost and Utilization Project (HCUP) AHRQ Healthcare Cost and Utilization Project (HCUP)

10. Variables Input prices Input prices – Price of labor – Price of capital Outputs Outputs – Admissions – Outpatient visits – Post-admission inpatient days Teaching status Teaching status – Binary variables for member of Council of Teaching Hospitals (COTH) and non-COTH hospital with at least one full-time equivalent (FTE) medical resident Time trend Time trend

11. A key challenge in applying SFA to health care settings is controlling for heterogeneity. A key challenge in applying SFA to health care settings is controlling for heterogeneity. Variables (continued)

12. Product mix Product mix – Acute care beds / total beds – Births / total admissions – ED visits / total outpatient visits – Outpatient surgical operations / total outpatient visits Outcome measures of quality Outcome measures of quality Patient mix Patient mix – Medicare Case Mix Index – Comorbidity variables

13. HCUP HCUP – A family of health care databases and related software tools and products developed through a Federal-State-Industry partnership and sponsored by AHRQ. – Includes State Inpatient Databases (SID), which contain the universe of inpatient discharge abstracts from participating states. Data from 24 states available to the public through the HCUP Central Distributor Data from 24 states available to the public through the HCUP Central Distributor Variables (continued)

14. The AHRQ Quality Indicators (QIs) are measures of health care quality that make use of readily available hospital inpatient administrative data, such as HCUP. The AHRQ Quality Indicators (QIs) are measures of health care quality that make use of readily available hospital inpatient administrative data, such as HCUP. Free software tools available online. Free software tools available online. Includes Includes – Inpatient Quality Indicators (IQIs), which reflect quality of care inside hospitals including inpatient mortality for medical conditions and surgical procedures. – Patient Safety Indicators (PSIs), which reflect quality of care inside hospitals, but focus on potentially avoidable complications and iatrogenic events.

15. Variables (continued) Quality measured by the application of the QI software to HCUP data. Quality measured by the application of the QI software to HCUP data. Analysis includes the following, risk-adjusted, in- hospital rates: Analysis includes the following, risk-adjusted, in- hospital rates: – Mortality for the following conditions Acute myocardial infarction (AMI), congestive heart failure (CHF), stroke, gastrointestinal hemorrhage, pneumonia Acute myocardial infarction (AMI), congestive heart failure (CHF), stroke, gastrointestinal hemorrhage, pneumonia – Failure to rescue – Iatrogenic pneumothorax – Infection due to medical care – Accidental puncture / laceration

16. Variables (continued) The Comorbidity Software assigns variables that identify comorbidities in hospital discharge records using the diagnosis coding of ICD-9-CM. The Comorbidity Software assigns variables that identify comorbidities in hospital discharge records using the diagnosis coding of ICD-9-CM. Available for free online. Available for free online.

17. Patient burden of illness controlled by the inclusion of hospital-level rates per discharge of the following comorbidities identified by the Comorbidity Software: Patient burden of illness controlled by the inclusion of hospital-level rates per discharge of the following comorbidities identified by the Comorbidity Software: Congestive heart failureCardiac arrhythmiasValvular disease Pulmonary circulation disordersPeripheral vascular disordersHypertension ParalysisOther neurological disordersChronic pulmonary disease Diabetes, uncomplicatedDiabetes, complicatedHypothyroidism Renal failureLiver diseasePeptic ulcer AIDSLymphomaMetastatic ulcer Solid tumor without metastasisRheumatoid arthritisCoagulopathy ObesityWeight lossFluid and electrolyte disorders Blood loss anemiaDeficiency anemiasAlcohol abuse Drug abusePsychosesDepression See Mutter et al. (2008) for details See Mutter et al. (2008) for details Variables (continued)

18. Inefficiency effects variables Inefficiency effects variables – Ownership – Medicare share of discharges – Medicaid share of discharges – Medicare HMO penetration rate – Hospital competition – Time trend See Rosko (2001) See Rosko (2001) Variables (continued)

19. SFA in the NHQR SFA is used to provide trends in hospital efficiency. SFA is used to provide trends in hospital efficiency. – This is a measure from the provider perspective. The measure first appeared in the 2007 NHQR, the first year there was an efficiency chapter. The measure first appeared in the 2007 NHQR, the first year there was an efficiency chapter. SFA became an endorsed measure in 2009. SFA became an endorsed measure in 2009.

20. Based on 1,368 urban, general, community hospitals from 26 states providing SID data Based on 1,368 urban, general, community hospitals from 26 states providing SID data – Represent 53% of all urban, general community hospitals 2001 – 2005 2001 – 2005 Follow estimation approach recommended by Rosko and Mutter (2008) Follow estimation approach recommended by Rosko and Mutter (2008) Last Year’s Analysis

21. Cost efficiency estimates converted to index numbers with a base of 100 for the year 2001 Cost efficiency estimates converted to index numbers with a base of 100 for the year 2001 – Places less emphasis on the specific magnitude of estimated cost efficiency Last Year’s Analysis (continued)

23. Ratios Ratios – Managers have relied on ratios that convey straightforward information. – Comparing SFA estimates with these ratios yields valuable insights into organizational performance. Last Year’s Analysis (continued)

24. MeasureEstimate Std. dev. Cost per case-mix-adjusted admission: Top quartile of hospital cost efficiency$4,340 1,087 Bottom quartile of hospital cost efficiency$6,241 2,350 Full-time equivalent employees per case-mix-adjusted admission: Top quartile of hospital cost efficiency.040 0.01 Bottom quartile of hospital cost efficiency.055 0.02 Average length of stay (days): Top quartile of hospital cost efficiency4.88 1.33 Bottom quartile of hospital cost efficiency5.22 1.80 Operating margin: Top quartile of hospital cost efficiency.033 0.13 Bottom quartile of hospital cost efficiency -.066 0.17 Last Year’s Analysis (continued)

25. Some findings from the parameter estimates: Some findings from the parameter estimates: – COTH hospitals are about 12 percent more expensive than non-teaching hospitals; minor teaching hospitals are nearly 4 percent more expensive. – Coefficients on infection due to medical care and accidental puncture / laceration were positive and significant. Occurrences of these patient safety events are costly to hospitals. Occurrences of these patient safety events are costly to hospitals. Zhan and Miller (2003) estimate they are associated with excess costs of $38,656 and $8,271, respectively. Zhan and Miller (2003) estimate they are associated with excess costs of $38,656 and $8,271, respectively. Last Year’s Analysis (continued)

26. Further findings from the parameter estimates: Further findings from the parameter estimates: – For-profit ownership, greater hospital competition, greater Medicare HMO penetration, and a higher share of Medicare discharges associated with increased cost-efficiency – Government ownership associated with reduced cost-efficiency Last Year’s Analysis (continued)

27. Looking ahead SFA estimates will appear in the 2009 NHQR. SFA estimates will appear in the 2009 NHQR.

28. Resources Copies of Rosko and Mutter (2008) and Mutter et al. (2008) are available in the back of the room. Copies of Rosko and Mutter (2008) and Mutter et al. (2008) are available in the back of the room. Further information on HCUP data, the Quality Indicator Software, and the Comorbidity Software are available at the HCUP and QI booths. Further information on HCUP data, the Quality Indicator Software, and the Comorbidity Software are available at the HCUP and QI booths.