2. Learning Objectives Describe the use of the Outcomes Impact Analysis model to assess economic impact of a CME activity Discuss the potential role of Outcomes Impact Analysis modeling in CME practice 2

3. What We’ve Been Hearing… “How do I convey the value of CME to non-CME administrators?” “I need to summarize an entire year of outcomes in a condensed format.” “I’ve been asked to justify my continuing medical education budget.” “How can I compare different outcomes from a number of activities?” 3

4. The Problems Confronting CME Funding and resources are decreasing – The core message of the value of CME is not reaching internal and external stakeholders, policy makers, and society – There are few resources for professionals to demonstrate the return on CME investment to nonmedical or noneducator audiences 4

5. Solutions Demonstrating the benefits of CME depends on effective communication Economic data provide a universally understood outcome measure— monetary impact Economic modeling of CME is a rapid, cost effective, and repeatable method of predicting cost impact 5

6. Outcomes Impact Analysis

7. Outcomes Impact Analysis: A Novel Analytic Framework Evaluating the economic impact of CME outcomes is based on the established principles of health economic modeling, decision analysis, probability theory, and statistical analysis No single existing economic analysis model is ideally suited for assessing CME outcomes 7

8. CME symposium was attended by 133 participants Reviewed strategies for blood conservation and implementation of the Society of Thoracic Surgeons Blood Conservation Clinical Practice Guidelines 93.8% were committed to change their practice based on what they learned Major barriers to blood conservation cited were cost, administration buy-in, and inertia Blood Conservation Symposium: Economic Implications 8

9. Outcomes impact analysis was used to estimate the potential economic impact of applying CME-related learning to clinical practice for cardiac surgical outcomes: – Prevention of bleeding-related complications – Prevention of reoperation for bleeding in coronary artery bypass graft (CABG) surgery Costs averted were estimated from the perspective of the healthcare provider Evaluating the Economic Impact 9

10. Bleeding-related complications (BRC) A study of 103,826 cardiac operations showed that mean hospital costs were $12,128 higher in patients with a BRC than in those without a BRC; the additive costs from 142,533 thoracic operations was $15,899 1 Reoperation for bleeding in CABG Nineteen studies studies evaluated additive costs of reoperation for bleeding following CABG (n=755,382); weighted mean was $24,048 2-4 Assumptions Operative volume – The STS/AATS Workforce Report showed that adult cardiac surgeons (n=1211) performed a mean of 155 cardiac operations and 214 thoracic operations in 2009 5 Models and Parameters 1. Stokes ME et al. BMC Health Serv Res. 2011;11:135-148. 2. Brown PP et al. Ann Thorac Surg. 2008;85:1980-1987. 3. Herwaldt LA et al. Infect Control Hosp Epidemiol. 2003;24:44-50. 4. Speir AM et al. Ann Thorac Surg. 2008;85:1980-1987. 5. Shemin RJ et al. Ann Thorac Surg. 2012;93:348-355. 10

11. Base Case Analysis 3 in 10 participants changed clinical practices as a result of CME learning in such a way as to prevent the outcome of interest (BRC or reoperation) Resulted in prevention of BRC and RFB in 2% and 1.5% of operations, respectively One year time period Base Case 11

12. One-way sensitivity analysis evaluated the potential savings for the base case when the proportion of participants who prevented outcomes of interest varied from 1 in ten to 5 in ten Probabilistic sensitivity analysis with second-order Monte Carlo simulation was used to evaluate robustness of the estimate to parameter uncertainty and to calculate a mean and confidence interval for the estimated value Adjustments Estimated values were standardized to 2012 $US using the medical care component of the Consumer Price Index Sensitivity Analysis 12

13. Results Model Estimated Value ($US 2012) Base CasePSA Mean (95% CI) Bleeding-related complications, cardiac1,500,112 1,502,769 (869,860–2,359,068) Bleeding-related complications, thoracic2,715,104 2,715,246 (1,590,308-4,217,092) Reoperation for bleeding, CABG2,230,8732,233,988 (1,223,901–3,648,719) Ravyn D et al. J Cont Educ Health Prof. 2014;34(S1):S41-S46. 13

14. Dynamic Estimator 14

15. Outcomes Impact Analysis StrengthsLimitations Rapid, cost effective, and highly versatile Can be used in funding decision making or to evaluate outcomes after an activity Estimates of the monetary return on education Does not require patient-level or medical record data Independent of educational modality, audience, or therapy area Allows outcomes comparisons across heterogeneous activities Data, scenario, assumptions, and time horizon are tailored to the decision maker’s needs Results easily understood by nonmedical, noneducator, and lay audiences Sensitivity analysis demonstrates results across a wide range of plausible scenarios Cost data may be unavailable or unreliable Does not provide direct evidence of patient impact; outcomes inferred from utilization Serves as an adjunct to, not a replacement of educational outcomes Does not usually consider indirect or intangible costs Limited to expenditures, does not examine quality of life Requires expertise in the therapy area, health services, outcomes, and economic modeling As with all economic predictive models, it may be necessary to validate estimated values after implementation 15

16. Conclusions Plausible economic estimates suggest that CME- related learning favorably impacting practice yields substantial cost savings Model prediction of averted costs associated with CME allows estimation of the economic impact on outcomes in the absence of patient-level outcomes data related to CME activities This model may also be used to optimize CME resources by identifying areas of educational need most likely to result in cost savings 16

17. For more information contact Dana Ravyn dravyn@cmeology.org