1. Critical Outcomes Report Analysis May 2008

2. Agenda Some Logistics Overview of why reports are wrong and how to fix them –This will help somewhat in reading them and in contracting for DM but critical outcomes report analysis is about learning how to read these things generally Sample Questions The Test

3. Logistics You can then either “officially” take the test or not. –There is no downside to taking it except that you can’t take it again for 6 months If you don’t want to take the test, you can either “play along at home” or if there is a group which wants to work together on the questions you can do that Or, you can go to the ROI precon next door (about 20% overlap in my slides) You can’t take the test home with you. You can request the non-test portion of the slides via email to diseasmgmt@aol.com or visit the website www.dismgmt.com and hit “contact us.”diseasmgmt@aol.comwww.dismgmt.com

4. If you pass… You may apply for individual certification for $500 for two years. You get listed on the DMPC website (see next page) announced on the listserve, and may be used as a professional credential –You will also be much better at reading these reports. Corporate certification is $2000

5. Beginning of List on Website IndividualCorporate Affiliation (note: Boldface indicates Corporate Certification in addition to individual) Ed Baas HealthMedia, Inc. Steve Bennett HealthMedia, Inc. David Brumley MD Blue Cross of Massachusetts John Charde MD Enhanced Care Initiatives, Inc. Brian Doran Quantum Health Inc. Laurie Doran Boston Medical Center Health Plan Thomas Hawkins Blue Cross of Massachusetts Sharon Hewner Independent Health Plan

6. Overview of Why Reports are wrong and how to fix them and be a hero to your organization…

7. …Rather than rely on others for your measurement

8. Reasons Why Reporting is often Wrong Look at these “checks and balances,” and ask yourself, why aren’t you already doing this in contracts with your vendor?

9. Plenty of Other Reasons too

10. Without further ado, three reasons reports are wrong: The Following don’t get done (except by DMPC-Certified Payors) The Dummy Year Analysis –The exact same methodology applied to a year in which you did not have disease management Plausibility Testing Critical Outcomes Report Analysis

11. Dummy Year Analysis Most contracts have a baseline period to which a contract period is compared (adjusted for trend) –Raise your hand if you don’t

12. Dummy Year Analysis Most contracts have a baseline period to which a contract period is compared (adjusted for trend) –Hand-raising time Watch what happens when you have a baseline and then compare a contract period (adjusted for trend) –Just the analysis, no program

13. In this Dummy Year Analysis example Assume that “trend” is already taken into account Focus on the baseline and contract period comparison

14. Base Case: Example from Asthma First asthmatic has a $1000 IP claim in 2005 2005 (baseline) 2006 (contract) Asthmatic #11000 Asthmatic #2 Cost/asthmatic

15. Example from Asthma Second asthmatic has an IP claim in 2006 while first asthmatic goes on drugs (common post-event) 2005 (baseline) 2006 (contract) Asthmatic #11000100 Asthmatic #201000 Cost/asthmatic What is the Cost/asthmatic In the baseline?

16. Cost/asthmatic in baseline? 2005 (baseline) 2006 (contract) Asthmatic #11000100 Asthmatic #201000 Cost/asthmatic$1000 Vendors don’t count #2 in 2005 bec. he can’t be found

17. Cost/asthmatic in contract period? 2005 (baseline) 2006 (contract) Asthmatic #11000100 Asthmatic #201000 Cost/asthmatic$1000$550

18. Base Case: How Dummy Year Analysis (DYA) fixes it 2005 (baseline) 2006 (contract) Asthmatic #11000100 Asthmatic #201000 Cost/asthmati c $1000$550 In this case, a “dummy population” falls 45% on its own without DM

19. So… If you were to do an asthma program the vendor should not get credit for the reduction that happens anyway –But they do –How do we know that? With a plausibility test, to be discussed later –First, some real-world Dummy Year Analyses (DYAs)

20. DYA real-world Result: Excerpt from Regence Blue Cross-DMPC study for Health Affairs released recently

21. DYA Result By Disease (using 1-year baseline and standard DMPC algorithms) – what is the difference which is caused automatically by just trending forward?

22. DYA Result in Wellness Source: Ariel Linden – citation On request

23. There was no program in this case – just two samplings and the average stayed the same Source: Ariel Linden – citation on request

24. Other evidence for Dummy Year Analysis (DYA) CMS studies – very carefully designed -- get results opposite those done without DYAs, and consistent with those done with DYAs Most Vendors oppose DYAs (and they aren’t in the DMAA guidelines) ROIs without DYA adjustment flunk plausibility testing…

25. Reason #2 The Dummy Year Analysis …Plausibility Testing Critical Outcomes Report Analysis

26. What is a plausibility test? You do it all the time…outside DM An easy way to directionally check results Measure total event rates for diseases being managed, like you’d measure a birth rate. Couldn’t be easier –Ask me for the specific directions. They’re free from DMPC (and now DMAA). See next page Example from previous asthma hypothetical

27. Event rates tracked by disease: the Plausibility Indicators Disease Program CategoryICD9s (all.xx unless otherwise indicated) Asthma493.xx (including 493.2x [1] ) [1] Chronic Obstructive Pulmonary Disease491.1, 491.2, 491.8, 491.9,. 492, 494, 496, 506.4 Coronary Artery Disease (and related heart- health issues) 410, 411, 413, 414 Diabetes250 Heart Failure428, 404.01, 404.03, 404.11, 404.13, 404.91, 404.93, 425.0, 425.4 [1] [1] 493.2x is asthma with COPD. It could fit under either category but for simplicity we are keeping it with asthma

28. Cost/asthmatic in contract period? 2005 (baseline) 2006 (contract) Asthmatic #11000100 Asthmatic #201000 Cost/asthmatic$1000$550

29. Asthma events in the payor as a whole 2005 (baseline) 2006 (contract) Asthmatic #11000100 Asthmatic #201000 Inpatient events/year 11

30. Plausible? How can you reduce asthma costs 45% without reducing planwide asthma event rate? Answer: You can’t. Not plausible

31. Several Examples of Plausibility Analysis Pacificare Some which didn’t turn out so well Plausibility-testing generally and benchmarks

32. PacifiCare HF Results

33. Several Examples of Plausibility Analysis Pacificare Some which didn’t turn out so well

34. Example of just looking at Diagnosed people: Vendor Claims for Asthma Cost/patient Reductions ER IP

35. What we did to plausibility-test… We looked at the actual codes across the plan This includes everyone Two years of codes pre-program to establish trend Then two program years

36. Baseline trend for asthma ER and IP Utilization 493.xx ER visits and IP stays/1000 planwide ER IP

37. Expectation is something like… 493.xx ER visits and IP stays/1000 planwide ER IP

38. Plausibility indicator Actual: Validation for Asthma savings from same plan including ALL CLAIMS for asthma, not just claims from people already known about 493.xx ER visits and IP stays/1000 planwide ER IP How could the vendor’s methodology have been so far off?

39. We then went back and looked… …at which claims the vendor included in the analysis…

40. We were shocked, shocked to learn that the uncounted claims on previously undiagnosed people accounted for virtually all the “savings” ER IP Previously Undiagnosed Are above The lines

41. Is it fair… To count the people the vendor didn’t know about?

42. You should be able to reduce visits in the known group by enough so that adding back the new group yields the reduction you claimed – otherwise you didn’t do anything ER IP Previously Undiagnosed Are above The lines

43. Applying Plausibility to Mercer presentation which found a “range” of possible savings in Respiratory DM Mercer’s view: “Varying the methodology has a significant impact on the results” Results “somewhere in that range” Our View: There is only one right answer and a Plausibility test will point to it

44. How Mercer could do a plausibility test on asthma Take two-three years of claims history in all primary-coded 493.xx claims for ER and IP Add together and divide by # of covered lives to get a rate Then Ask: What happens in the program year?

45. Possible trend prior to program

46. For the program to have saved $6-million, this indicator would have to plunge (it didn’t)

47. Let’s Macro-Plausibility-Test Wellness The Dummy Year Analysis Plausibility Testing –For Wellness Critical Outcomes Report Analysis

48. Macro Plausibility for Wellness Here’s how you know wellness reports are inflated or impossible Compare all these reported dramatic results in smoking cessation and weight loss to CDC statistics for the US as a whole –Even as most large (and many smaller) companies are “producing” these results, obesity continues to climb and the drop in adult smoking rates has stalled

49. October 26, 2006 Drop in Adult Smoking Rate Stalls THURSDAY, Oct. 26 (HealthDay News) -- The number of adult smokers in the United States did not change from 2004 to 2005, suggesting that the decline in smoking over the past seven years has stalled, a new federal report found. In 2005, 45.1 million adults, or 20.9 percent, were cigarette smokers – 23.9 percent of men and 18.1 percent of women. In addition, 2.2 percent of U.S. adults were cigar smokers and 2.3 percent used smokeless tobacco, according the report. "After years of progress, what we are seeing is no change in adult prevalence of smoking between 2004 and 2005," said report author Terry Pechacek, the associate director for science at the U.S. Centers for Disease Control and Prevention's Office on Smoking and Health.

50. Obesity Trends* Among U.S. Adults BRFSS, 1985 (*BMI ≥30, or ~ 30 lbs. overweight for 5’ 4” person) No Data <10% 10%–14%

51. Obesity Trends* Among U.S. Adults BRFSS, 1987 (*BMI ≥30, or ~ 30 lbs. overweight for 5’ 4” person) No Data <10% 10%–14%

52. Obesity Trends* Among U.S. Adults BRFSS, 1989 (*BMI ≥30, or ~ 30 lbs. overweight for 5’ 4” person) No Data <10% 10%–14%

53. Obesity Trends* Among U.S. Adults BRFSS, 1993 (*BMI ≥30, or ~ 30 lbs. overweight for 5’ 4” person) No Data <10% 10%–14% 15%–19%

54. Obesity Trends* Among U.S. Adults BRFSS, 1997 (*BMI ≥30, or ~ 30 lbs. overweight for 5’ 4” person) No Data <10% 10%–14% 15%–19% ≥20%

55. Obesity Trends* Among U.S. Adults BRFSS, 2001 (*BMI ≥30, or ~ 30 lbs. overweight for 5’ 4” person) No Data <10% 10%–14% 15%–19% 20%–24% ≥25%

56. Obesity Trends* Among U.S. Adults BRFSS, 2005 (*BMI ≥30, or ~ 30 lbs. overweight for 5’ 4” person) No Data <10% 10%–14% 15%–19% 20%–24% 25%–29% ≥30%

57. Obesity Trends* Among U.S. Adults BRFSS, 2006 (*BMI ≥30, or ~ 30 lbs. overweight for 5’ 4” person) No Data <10% 10%–14% 15%–19% 20%–24% 25%–29% ≥30%

58. Agenda Some Logistics Overview of why reports are wrong and how to fix them –This will help somewhat in reading them and in contracting for DM but critical outcomes report analysis is about learning how to read these things generally Sample Questions The Test

59. Sample Question #1 Look at each of these slides and both together to find major reporting concerns if any

60. Table 1: Inpatient Impact of Program (Year One) DiseaseBaseline IP days/1000 Program IP days/1000 Change Asthma996747-25% CAD18971391-27% CHF97228581-29% COPD25122151-14% Diabetes15341522-1%

61. Table 2: Impact on Physician Visits DiseaseBaseline MD visits/1000 Program MD Visits/1000 Change Asthma69905907-15% CAD88298580-3% CHF78767506-5% COPD84818090-4% Diabetes79277737-2%

62. What you might have noticed – first slide No plausibility test for very high utilization reduction Asthmatics don’t have 996 days per 1000 –Not clear whether they are referring to days per 1000 disease members or days per 1000 overall (either way, it’s wrong) Nor does CHF have so many days per 1000 CHF days did not decline 29%

63. Second slide, and both combined Ridiculously high number of doctor visits Doctor visits should be going up or staying the same, not going down –This suggests strongly that a DYA is needed because they seem to have selected a high- using sample as a baseline No correlation between MD-intensity and IP-intensity of diseases

64. Sample #2: Comment on these CHF measures

65. Sample #3: Improvement in Plan A of HEDIS Scores: Why is/isn’t this a valid improvement? HEDIS EFFECTIVENESS OF CARE MEASURES Commercial200320042005 Controlling High Blood Pressure62.266.868.8 Beta blocker after AMI69.872.577.7 Diabetes: HbA1c Testing84.686.587.5 Diabetes: Lipid Control (<100 mg/dL)34.740.243.8 Medical Assistance with Smoking Cessation68.669.671.2 Medicare200320042005 Controlling High Blood Pressure61.464.666.4 Beta blocker after AMI92.99493.8 Diabetes: HbA1c Testing87.989.188.9 Diabetes: Lipid Control (<100 mg/dL)41.947.550 Medical Assistance with Smoking Cessation63.364.775.5

66. Asthma Plausibility Test Baseline vs PY01 Program Year BaselinePY01Variance Net Paid$6,671,855$9,656,95944.7% Events3,4164,34627.2% Days3,8755,18333.8% Risk MM's874,8781,245,78342.4% PMPM$7.63$7.751.6% Events / 100046.8541.86-10.7% Days / 100053.1549.93-6.1% Cost / Day$1,722$1,8638.2% Sample #4: Does this one pass the Sniff test?

67. Sample #5: Does this pass the sniff test for diabetes?

68. Sample #6 small group bid Comment on this bid for a group of 80,000 people

70. Agenda Some Logistics Overview of why reports are wrong and how to fix them –This will help somewhat in reading them and in contracting for DM but critical outcomes report analysis is about learning how to read these things generally Sample Questions Break The Test

71. Test Overview Answer each question by number by saying what’s wrong or indicating that it can be concluded, based on the data provided, that nothing major is obviously wrong. Keep it concise. Don’t just automatically say no DYA or plausibility test Scoring: 3 points for each item found which DMPC missed 2 points for each major item found 1 point for each minor item and watch-out found 0 points for each item where there was none -1 point for each item found which were really OK enough to be plausible but which were identified

72. Answer Sheet (if you are taking the test and want to be scored) Name_____________ Organization_____________ Email________________ Phone_______________ Make sure to number each question and put the sheets in order on top of this one And just in case they get separated put your NAME or identifier on each page. Then clip them together at the end using the handy clip provided

73. Question 1 – comment on this website

74. Question #2 In the following example, utilization figures were multiplied by the (assume to be correct) cost figures to get a savings –Note that the savings is the difference between the two bars Assume (correctly) no other changes were talking place

75. Savings by Category of Utilization per 1000 members per month (2004 vs. 2003) (note: The difference between the bars is the savings)

76. Question 3 Assume on the next slide that the admission reductions are calculated validly and are the result of the program

77. Question #3: Comment on the plausibility of this Cigna report (assume a reasonable valid methodology was used to calculate admission reduction) Disease Category All-cause Admission Reduction per disease member All-cause Claims Cost Reduction per disease member Asthma2%12% cardiology5%15%

78. Question 4 Comment on the Indiana Medicaid results

79. Indiana Medicaid CHF Study Group vs. Usual Care Total N = 186 Issue-Spotter #4: What is wrong with this slide Overall savings of $758 PMPM

80. Question #5 Comment on these results reported to a major employer (assume here as in all cases that low-risk and high-risk sum to the total managed population AND that these are asthma-specific changes)

81. Asthma Hospital Days and Admissions -48% -70% -43% DAYS ADMISSIONS

82. Question #6 The next two slides with all-in admissions and ER visits are from the same payor, same study –Find a major issue(s) which invalidates the result or indicate that the result is probably reasonably valid “R#1” and “R#2” refer to reporting periods of one year each

83. CHF Group #1 Emergency Room Visits/Year Total N = 1166 High Risk N = 268 Low Risk N = 898

84. CHF Group #1 Inpatient Admissions/Year Total N = 1166 High Risk N = 268 Low Risk N = 898

85. CHF Group #1 Inpatient Admissions/Year Total N = 1166 High Risk N = 268 Low Risk N = 898

86. Question #7 Find the mistake(s) if any (assume inflation adjustment is done correctly)

87. Pre-post comparison: Asthma Medicaid Disabled Population Baseline Period 1/03- 12/03 paid through 6/30/04 Study Period 1/04- 12/04, paid through 2/28/05 Member- months 1504731884 PDMPM$432$391 Gross savings & ROI $2,400,125 2.72 – to -1

88. Question #8 Comment on multiple issues on the following two slides representing the same study. Notes: –“Core Conditions” are the sum of the conditions above the line –“Extended Conditions” are managed conditions other than the Core Conditions –“Care Support” is disease managed group –Under each of the 3 categories, the two columns are comparisons between the baseline and reporting periods for the study and concurrent control groups

89. Cohort Study Results (all claims, all members)

90. ROI and PMPM reductions at 6 Months Reporting Period –July - December 2002 Base Period –July - December 2001 Total ROI 2.48 : 1 –Extended Conditions 4.23 : 1 –Core Conditions 1.86 : 1 “Our Auditors validated a $42 PMPM reduction due to this program”

91. Combined Reporting Period –July - December 2002 Base Period –July - December 2001 Total ROI 2.48 : 1 –Extended Conditions 4.23 : 1 –Core Conditions 1.86 : 1 Auditors validated a $42 PMPM savings

92. Sidebar Note Even though the previous slides were published I am not using the name because it wouldn’t be fair to the health plan which has subsequently dramatically improved its methodology(ies) –So if you recognize it don’t hold it against them. They would win a “most improved measurement” award

93. Question 9 Comment on the likely validity of the following slide

94. Program Year One – Clinical Indicators Clinical Outcomes:

95. Question #10 Comment on the following slide – CAD disease management program

96. Top Ten 2003 Diagnoses—admissions per 100 Cardio Disease Management Members (pre- and post-DM – savings is difference) “Symptoms” really is a nICD9 code

97. Question 11—Comment on CT Medicaid Current RFP May be a little hard to read I will display on Word

99. Question 12: Comment on this release IRVING, Texas--(BUSINESS WIRE)--Nov. 18 --A pediatric asthma disease management program offered by AdvancePCS saved the State of North Carolina nearly one-third of the amount the government health plan expected to spend on children diagnosed with the disease

100. Question 13: Comment on validity of this statement by a major commercial health plan “Over a 10-year period, we have reduced the rate of heart attacks by 5 per 100 people”