1. CMS SAS Users Group Conference Learn more about THE POWER TO KNOW ® October 17, 2011 Medicare Payment Standardization Modeling using SAS Enterprise Miner Michelle Roozeboom, PhD & Brian O’Donnell, PhD Buccaneer, A General Dynamics Company

2. Chronic Condition Warehouse Research database containing patient- centric data – 1999-current: 100% Medicare and Medicaid data linked across the continuum of care – All CCW data files linked by assignment of a unique, unidentifiable beneficiary link key (BENE_ID) for ease of analysis across data files (unique link keys replace health insurance claim (HIC) numbers

3. Background of Geographic Variation Database Uses the CCW as source data Created under a contract with the Policy Data Analysis Group Standardized Medicare’s payment amounts to remove geographic differences and hospital characteristic differences in payment rates for individual services Service Utilization Files are aggregated to produce the Beneficiary Service Level Files which contain one record per beneficiary for each service classification

4. Current Standardization Process Institutional (Part A) standardization – Determine a base rate and multiply that rate by weights determined by the particular service performed (e.g., DRG weights) – Adds information for other relevant factors, such as geographic outlier payments – Recalculate the payment amount, removing the geographic factors such as: wage index factors, medical teaching status, and Disproportionate Share Hospital (DSH) factors

5. Current Standardization Process Non-Institutional (e.g., Part B and hospital outpatient services) standardization – Link information on the claim with the appropriate fee schedule – Derive a national rate as opposed to applying a geographically specific rate

6. SAS Enterprise Miner Project Overview

7. Graph Explore: Review of Part A Data

10. Data Preparation 2008 Part A service files were used for analysis – Limited to Acute Inpatient Services – 20% claim sample was used – Actual payment was “target” variable – Included an identifier for acute services for Maryland providers

11. Data Preparation

12. Graph Explore Output Screen

13. Stat Explore Output Screen

14. Strength of Relationship with Dependent Variable

15. Correlation With Dependent Variable

16. Categorical Variables

17. Variability for Categorical Variables

18. Regression Analysis Linear Regression with actual payment as dependent variable Stepwise regression was used to fit the model Model was fitted both with and without interactions

19. Regression Output Screen – No Interactions

20. Plots of Mean Predicted Payment Compared to Actual Payment

21. Plots of MaximumPredicted Payment Compared to Actual Payment

22. Regression ANOVA table Analysis of Variance Sum of Source DF Squares Mean Square F Value Pr > F Model 41 1.6170343E13 394398598668 8663.89 <.0001 Error 919453 4.1855424E13 45522092 Corrected Total 919494 5.8025767E13 Model Fit Statistics R-Square 0.2787 Adj R-Sq 0.2786 AIC 16214148.624 BIC 16214150.628 SBC 16214641.350 C(p) 42.0000

23. Model Fit Statistics Statistics LabelTrainValidationTest Akaike's Information Criterion 16,214,149 Average Squared Error 45,520,01346,213,60845,574,862 Average Error Function 45,520,01346,213,60845,574,862 Degrees of Freedom for Error 919,453 Model Degrees of Freedom 42 Total Degrees of Freedom 919,495 Divisor for ASE 919,495689,621689,622 Error Function 41,855,424,402,17131,869,874,606,09831,429,427,255,852 Final Prediction Error 45,524,172 Maximum Absolute Error 411,742979,415311,907 Mean Square Error 45,522,09246,213,60845,574,862 Sum of Frequencies 919,495689,621689,622 Number of Estimate Weights 42 Root Average Sum of Squares 6,7476,7986,751 Root Final Prediction Error 6,747 Root Mean Squared Error 6,7476,7986,751 Schwarz's Bayesian Criterion 16,214,641 Sum of Squared Errors 41,855,424,402,17131,869,874,606,09831,429,427,255,852 Sum of Case Weights Times Freq 919,495689,621689,622

24. Variable Significance Type 3 Analysis of Effects Sum of Effect DF Squares F Value Pr > F CLM_IP_ADMSN_TYPE_CD 5 4.0932E11 1798.34 <.0001 CLM_PASS_THRU_PER_DIEM_AMT 1 2.38121E11 5230.89 <.0001 CLM_SRC_IP_ADMSN_CD 10 2.09188E11 459.53 <.0001 CLM_UTLZTN_DAY_CNT 1 4.76459E12 104665 <.0001 IP_COINS 1 1.88776E10 414.69 <.0001 IP_DED 1 3.95144E10 868.03 <.0001 NCH_CLM_TYPE_CD 1 6.05783E11 13307.4 <.0001 NCH_DRG_OUTLIER_APRVD_PMT_AMT 1 1.37552E12 30216.4 <.0001 PTNT_DSCHRG_STUS_CD 18 2.30196E11 280.93 <.0001 TRANSFER 1 522523298 11.48 0.0007 MD 1 1.14579E11 2516.99 <.0001

26. Regression Output Screen – Interactions Included

27. Regression Output – Interactions Included Analysis of Variance Sum of Source DF Squares Mean Square F Value Pr > F Model 376 1.6694034E13 44399026829 987.33 <.0001 Error 919118 4.1331733E13 44968908 Corrected Total 919494 5.8025767E13 Model Fit Statistics R-Square 0.2877 Adj R-Sq 0.2874 AIC 16203241.402 BIC 16203243.710 SBC 16207664.207 C(p) 377.7173

28. Model Fit Statistics – Interactions Included Statistics LabelTrainValidationTest Akaike's Information Criterion 16,203,241 Average Squared Error 44,950,47045,901,18245,230,501 Average Error Function 44,950,47045,901,18245,230,501 Degrees of Freedom for Error 919,118 Model Degrees of Freedom 377 Total Degrees of Freedom 919,495 Divisor for ASE 919,495689,621689,622 Error Function 41,331,732,859,83931,654,419,084,52631,191,948,490,416 Final Prediction Error 44,987,346 Maximum Absolute Error 372,404981,944308,056 Mean Square Error 44,968,90845,901,18245,230,501 Sum of Frequencies 919,495689,621689,622 Number of Estimate Weights 377 Root Average Sum of Squares 6,7056,7756,725 Root Final Prediction Error 6,707 Root Mean Squared Error 6,7066,7756,725 Schwarz's Bayesian Criterion 16,207,664 Sum of Squared Errors 41,331,732,859,83931,654,419,084,52631,191,948,490,416 Sum of Case Weights Times Freq 919,495689,621689,622

29. Validation Dataset Results Data Role=VALIDATE Target Variable=ACTUAL_PMT Number of Mean Mean Percentile Observations Target Predicted 5 34558 18044.32 18205.80 10 34586 11077.23 10876.85 15 34615 9812.84 9506.95 20 34317 8794.13 8720.79 25 34334 8468.74 8150.62 30 34553 7506.47 7702.61 35 35017 7028.92 7271.02 40 45881 6745.32 6847.69 45 23859 6086.08 6581.16 50 33133 5653.99 6235.69 55 39907 5833.69 5866.46 60 29029 5491.00 5542.13 65 34465 5215.32 5220.52 70 34732 5102.76 4963.06 75 37436 4839.71 4619.78 80 46742 4323.79 4289.00 85 19014 4140.28 4046.06 90 35845 3955.34 3690.52 95 37874 3492.68 3298.39 100 29724 982.72 1255.64

30. Scored Distribution

31. Scoring Results

32. SAS code Tab

33. Comparison Model calculated standardized payments were compared to standardized payments resulting from the current method Difference in standardized payments was calculated as Standardized Difference = Standardized Payment – Predicted Actual Payment

34. Summary Statistics VariableMedianMinimumMaximumMean Standard DeviationSkewnessKurtosisAbs C.V. Coefficient of Variation CLM_NON_U TLZTN_DAYS_ CNT00805.000.153.56143.3428,525.3323.03 IP_COINS0038,160.0042.15676.9133.561,473.7016.06 NCH_DRG_O UTLIER_APRV D_PMT_AMT00611,466.50345.034,783.6245.813,835.6513.86 stand_diff-899.95-177,858.43378,350.41-588.325,295.215.22330.019.00-9.00 CLM_PASS_T HRU_PER_DI EM_AMT0.00 1,796.7712.0346.075.5966.093.83 abs_stand_dif f2,040.180.18378,350.413,131.594,310.2713.56707.551.38 ACTUAL_PMT5,638.460.00601,345.037,409.538,108.0910.35412.251.09 IP_DED1,024.000.001,024.00708.95472.23-0.84-1.300.67 Predicted_Ac tual_PMT5,745.67-12,164.14222,994.036,541.884,174.259.68252.390.64

35. Correlation with Difference in Standardized Payments Claim Utilization Day Count DRG Outlier Approved Payment Amount IP DeductibleIP Coinsurance -0.147-0.133-0.0134-0.1057

36. Variability of Categorical Variables

37. Evaluation of Standardized Differences

41. Conclusions and Lessons Learned A multilinear model without interactions and polynomial terms closely approximates a standardized payment based on payment rules for non-extreme payments A more accurate model may require splicing of data for high and low payments Large differences between standardized payments may be related to payments from Maryland providers, claim type

42. Conclusions and Lessons Learned Enterprise Miner allows for graphical evaluation of results for each step of a modeling process Different model parameters can be changed and applied quickly and easily Measurement scale for variables must accurate or results may be inaccurate or not produced

43. Conclusions and Lessons Learned Class variables should have a maximum of 512 categories Regression results will still be produced regardless of underlying assumptions – Assumptions should still be tested and evaluated to ensure accuracy of results

44. Questions