Presentation is loading. Please wait.

Presentation is loading. Please wait.

Development and Evaluation of CMS-HCC Concurrent Risk Adjustment Models Presented by Eric Olmsted, Ph.D. Gregory Pope, M.S. John Kautter, Ph.D. RTI International.

Similar presentations

Presentation on theme: "Development and Evaluation of CMS-HCC Concurrent Risk Adjustment Models Presented by Eric Olmsted, Ph.D. Gregory Pope, M.S. John Kautter, Ph.D. RTI International."— Presentation transcript:

1 Development and Evaluation of CMS-HCC Concurrent Risk Adjustment Models Presented by Eric Olmsted, Ph.D. Gregory Pope, M.S. John Kautter, Ph.D. RTI International Presented at Academy Health June 26, 2005 411 Waverley Oaks Road Suite 330 Waltham, MA 02452-8414

2 Concurrent Risk Adjustment Introduction Overview Risk Adjustment/HCC Model Concurrent v. Prospective Project Goals and Challenges Model Development Model Evaluation Summary and Conclusion

3 Overview Risk Adjustment Introduction Population Risk Adjustment: The process by which the health status of a population is taken into consideration when setting capitation rates or evaluating patterns or outcomes of practice Risk adjustment is used to create apples to apples comparisons Risk adjustment removes the effect of health status differences Reduces or eliminates the problem of selection

4 Overview Risk Adjustment Model Model calibrated on 5% national sample of Medicare fee- for-service beneficiaries Expenditures are regressed on HCC (& demographic) risk markers to estimate incremental impact of each diagnosis on expenditures Annualized Expenditures = Σα i + Σβ i + Є i α i = demographic markers β i = HCC markers

5 Overview HCC Model Full model contains 184 HCCs CMS-HCC model contains 70 HCCs CMS-HCCs: Cover a broad spectrum of health disorders Have well-defined diagnostic criteria Exclude highly discretionary diagnoses Include conditions with significant expected health expenditures Demographic Markers Age, Gender, Medicaid, & Originally Disabled Status Ensure means for demographic populations correctly estimated

6 Overview Concurrent vs. Prospective Prospective risk adjustment uses current year diagnoses to predict next years expenditures Chronic conditions are more important Concurrent risk adjustment uses current year diagnoses to predict this years expenditures Acute conditions are more important

7 Overview Concurrent vs. Prospective AMI: Prospective Coefficient = $1,838 Concurrent Coefficient = $12,211 63% of HCC coefficients with >$1,000 difference R-squared: Concurrent -.4811 Prospective -.0981

8 Project Goals Concurrent Risk Adjustment Project Goals: Develop payment model for Pay-for-Performance demonstration Develop model for use in profiling physicians Make model consistent with prospective CMS-HCC model that is being used for MA payment, and its data collection requirements Improve prediction across the spectrum of patient cost

9 Concurrent Modeling Challenges Applied standard HCC model Resulted in negative predictions and coefficients Concurrent HCC coefficients fit high-cost beneficiaries This forces age-sex coefficients down and they sometimes become negative Age-sex coefficients reflect the average beneficiary Negative age-sex coefficients can lead to negative predictions

10 Model Challenges Standard Regression

11 Model Challenges Split Sample Regression

12 Model Challenges Regression through the Origin

13 Model Challenges Nonlinear Regression

14 Project Goals Model Selection Criteria for Model Selection Avoid negative predictions, which lack face validity Avoid negative coefficients Maintain correct age-sex means to prevent age and sex selection by providers Prefer simple models to complex models Select model with good performance among model evaluation measures

15 Model Development Sample Statistics 1.4 million FFS Medicare beneficiaries with mean expenditures of $5,214 Beneficiaries with at least one CMS-HCC represent 61% of the population, but provide 94% of all Medicare expenditures

16 Model Development Standard Models Full HCC Model 184 HCCs & demographics CMS-HCC Model 70 HCCs & demographics Interaction and Topcoding Models Created disease and demographic interactions to tease out high-expense beneficiaries Created topcoded models to reduce impact of outliers

17 Model Development Alternative Models Nonlinear Models Log model Square root model Split Sample Models Designed separate models for populations with different expected expenditures Community/Institutional High Cost/Low Cost HCC Catastrophic HCC Multi-stage models including two-part and four-part logit models Simple two-stage model with demographic multipliers Segmentation

18 Model Evaluation Standard Model Results Full HCC model suffers not only from 30% negative predictions, but also contains negative HCC coefficients CMS-HCC model explains 92% of the variation that the Full HCC model explains CMS-HCC model eliminates negative HCC coefficients CMS-HCC model has only 10% negative predictions Interaction and Topcoding Models Did not sufficiently reduce negative predictions

19 Model Evaluation Alternative Model Results Nonlinear Models Log model and square root model did not produce reasonable predictions Split Sample Models Splitting sample by community/institutional did not eliminate negative predictions Splitting sample by disease burden eliminated negative predictions

20 Model Evaluation Measures of Model Performance R 2 within.04 for all models R 2 did not differentiate models Predictive Ratio = Average of models predictions Average of actual expenditures Where each of the two averages is taken over the individuals in the subgroup Predicted expenditure deciles Number of HCCs for a beneficiary

21 Model Evaluation Predictive Ratios by Expenditure Percentile


23 Model Evaluation Predictive Ratios by # of HCCs


25 Concurrent Model Evaluation Model Summary High Cost & Catastrophic Models performs well Some face validity problems with splitting HCCs into high-cost and low-cost Still has negative predictions Four Part Model also performs well Computationally advanced and hard to interpret intuitively No negative predictions Sample Segmentation Model performs very well Also computationally advanced Two-Stage Multiplier Model performs adequately No face validity problems

26 Concurrent Model Evaluation Conclusion Nonlinearities cause difficulties in concurrent risk adjustment model calibration Negative coefficients and predictions These difficulties can be addressed with: Nonlinear models Split sample models But nonlinear/split sample models add complexity Difficult to estimate Difficult to interpret Adds instability Two-Stage Multiplier Model Good face validity, avoids negative coefficients and predictions Simpler to estimate and interpret

Download ppt "Development and Evaluation of CMS-HCC Concurrent Risk Adjustment Models Presented by Eric Olmsted, Ph.D. Gregory Pope, M.S. John Kautter, Ph.D. RTI International."

Similar presentations

Ads by Google