1. GETTING YOUR MONEY’S WORTH: RETURN ON INVESTMENT Craig Thomas, CDC-OSTLTS Greg Randolph, North Carolina Center for Public Health Quality Brynn Riley, Maine Department of Health

2. Craig Thomas, PhD Division of Public Health Performance Improvement Office for State, Tribal, Local and Territorial Support Centers for Disease Control and Prevention Open Forum for Quality Improvement June 20, 2012 Measuring the Impact of Improvement: Lessons from the National Public Health Improvement Initiative (NPHII) Office for State, Tribal, Local, and Territorial Support Division of Public Health Performance Improvement

3. National Public Health Improvement Initiative Five year initiative for health departments  Adopt and institutionalize cross-cutting performance management and quality improvement methods  Improve accountability, efficiency, and effectiveness of public health programs and services  Advance health departments’ readiness for public health accreditation

4. NPHII Return On Investment Workgroup  Why is ROI for improvement important?  Greater accountability in times of scare resources  Stronger evidence base for what works to improve business practices and service delivery for greater health impact  Promotes a business culture of improvement in public health  Purposes:  Identify practical approaches to measuring the costs and benefits of improvement with select NPHII grantees  Provide technical assistance as requested  Develop and share ROI methods, tools, and approaches

5. Analytic Approaches to ROI  Cost Benefit Analysis  QI methods on health outcomes is important from societal perspective  Not always feasible  Budget Impact Analysis  Offers practical approach to assessment of the costs and benefits of QI methods Calculates the cost and benefits of improving business processes and service delivery Focused on efficiency and effectiveness outcomes Documents both tangible and intangible benefits of QI

6. Key QI Outcome Measures  Efficiency: Doing something well with the least amount of waste  Time saved  Money saved  Reduced steps in a process  Effectiveness: Achieving a purpose or desired result  Quality enhancement (e.g., completeness and accuracy of data)  Increased customer service satisfaction  Increased reach of services  Implementation of evidence-based practices

7. Other QI Outcome Measures  Improvements that relate to both efficiency and effectiveness:  Consolidation and sharing of services  Leverage of additional funds / revenue streams  Integration of networks  Pilot testing and evaluation of innovative ideas

8. Overview of ROI Methodology for QI  Planning:  Develop objectives; ‘AIM’ Statement, measurement and data collection plans  Data Collection:  Establish baseline, collect cost data and outcomes before, during, and after implementation of improvement effort  Data Analysis:  Control for alternative hypotheses; convert hard date (e.g., time and staff) to monetary value; calculate the ROI (net project benefits/project costs) times 100  Identify Intangible Benefits:  Increased customer satisfaction; improved buy-in and organization support; greater credibility within the community

9. NPHII: Strategies to Increase Efficiencies Out of 74 grantees, the following have prioritized strategies to target efficiencies PercentFocus 70Decreasing cost to deliver services 69Decreasing time to deliver services 57.5Decreasing time to award contracts 54Decreasing staff allocation to deliver services

10. NPHII: Increasing Efficiencies through QI  Early examples in saving time and money  Lab data reporting to CDC: Reduced lag time from 14-21 days to 2–3 days  IT costs: Identified strategies to save over $1.2 million annually

11. NPHII: Strategies to Improve Effectiveness Out of 74 grantees, the following are prioritizing strategies to strengthen effectiveness PercentFocus 94.4Using data to prioritize activities for maximum impact 94.5Identifying and implementing evidence-based practices 92Engaging in cross-jurisdictional partnerships 93Using an organization-wide performance management system

12. NPHII: Improving Effectiveness thru QI  Early successes achieved by some grantees  Increased enrollment in Medicaid family planning program by 109 percent in one year  Streamlined clinic flow of family planning visits to improve patients’ experience  Increased referrals of pregnant women in WIC to smoking cessation programs

13. For more information please contact Centers for Disease Control and Prevention 1600 Clifton Road NE, Atlanta, GA 30333 Telephone, 1-800-CDC-INFO (232-4636)/TTY: 1-888-232-6348 E-mail: cdcinfo@cdc.gov Web: www.cdc.gov The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention. Phillips, J., & Phillips, P. (2007). Show me the Money: The Use of ROI in Performance Improvement, Part 1. Performance ROI in Performance Improvement, Part 1. Performance Improvement; V. 46, No 9. Improvement; V. 46, No 9. OSTLTS Performance Management / Quality Improvement Resources: http://www.cdc.gov/stltpublichealth/Performance/index.html Office for State, Tribal, Local and Territorial Support Division of Public Health Performance Improvement Resources

14. Application of ROI to Quality Improvement Projects Greg Randolph MD, MPH NC Center for Public Health Quality

15. Why is ROI Important? Need to improve performance given substantial burden of population health issues and health disparities in US PH agencies are increasingly applying QI methods to their work – involves a substantial upfront investment Sound business practices can help in a time of fiscal scrutiny and dwindling budgets/resources

16. How We Measure and Use ROI… ROI Model Template linked to aim statement’s measurable goals (see handout) Communications

17. 17 ROI Process Model Planning Develop AIM and Measurement Plan ------------------ Think about Potential Business Impact Data Collection Before Improvements (Baseline) ------------------ After Improvements ------------------- Capture costs throughout Data Analysis More Clues Waste identified via VSM and observation ------------------ QI Tools used for improvements Convert Measurement and Benefits Data to Monetary Values Calculate ROI Using Cost and Benefit Values Identify Intangible Benefits Generate Report with Conclusions and Lessons Learned Communicate Results to Target Stakeholders Reporting Lou Anne Crawley-Stout, MBA, CLSSBB, PMP Adapted from “ROI at Work” by Jack J. Phillips and Patricia P. Phillips

18. Terminology and Formula 18 ROI (return on investment): A performance measure used to evaluate the efficiency of an investment. ROI = Benefits-Costs/Costs EI (economic impact): Refers to costs and benefits of an activity. EI = Benefits-Costs Example: NC Institute for Public Health (NCIPH) Project Benefits (annualized savings)Project Costs Archiving (paper/printing) $ 360Staff time/travel $ 9850 Rent elimination 1490Archiving supplies 200 Labor/time savings 16330 $10050 Overtime reduction 2300 $20480 ROI: $20480-10050/10050 = $1.04 For every $1 invested in QI, NCIPH received in return $1.04 after costs. EI: $20480-10050 = $10430

19. Communication Tips Do stakeholder analysis early and plan for when/how/what will communicate to stakeholders – customize to specific stakeholder audience – Since ROI relatively new for public health, best to report face- to-face Stress that ROI data should not be used to reduce budgets further – by cutting budget of organization generating substantial ROI, would starving most innovative and effective organizations and halt spread of use of ROI Communicate how ROI illustrates you are better able to achieve your mission and other intangibles

20. Lessons Learned “Begin with the end in mind” – Begin with your aim statement – Capture tangibles and intangibles throughout When converting tangible benefits into monetary values, be conservative – Vet with your team and selected stakeholders Intangible Benefits (non-monetary benefits) are extremely valuable

21. http://journals.lww.com/jphmp/toc/2012/01000 Additional ROI Results in Jan / Feb 2012 issue of Journal of Public Health Management & Practice “Applying Lean Principles and Kaizen Rapid Improvement Events in Public Health Practice” http://journals.lww.com/jphmp/toc/2012/01000 21 Resources

22. Resources ROI at Work: Best-Practice Case Studies from the Real World; Jack J. Phillips and Patricia Pulliam Phillips www.ncpublichealthquality.org www.ies.ncsu.edu/ Greg Randolph: greg.randolph@ncphf.org

23. Brynn Riley Performance Improvement Manager, Maine June 20, 2012

24.  Quality Improvement (QI) teams have an opportunity to show projects’ return on investment (ROI) to stakeholders  QI teams should consider the:  upfront investment before applying QI methods  costs and benefits throughout the project’s duration.

25.  Accounting for costs and benefits will  Help improve the organization’s case for undertaking a QI project  Help prioritize competing interests  Help account for community or societal benefits where data doesn’t exist  Include intangible benefits, which are a significant and growing part of an organization’s and community’s worth. Developed by B. Riley - Brynn.Riley@maine.gov and J. Moran – jmoran@phf.org 6/10/12

26.  Consider how it translates into public health and community value.  Part of public health is preventing adverse health outcomes for a population, essentially creating “non- events”.  Including intangible benefits in cost benefit calculations account for the organizations added value to the community that may not otherwise be included.  Concerns:  Intangible benefits cannot be easily measured in dollar terms  The benefits may not be viewed as valid by some people Developed by B. Riley - Brynn.Riley@maine.gov and J. Moran – jmoran@phf.org 6/10/12

27.  Use the AIM statement as a springboard to hone the a QI Project’s focus and evaluate return on investment.

28. Developed by B. Riley - Brynn.Riley@maine.gov and J. Moran – jmoran@phf.org 6/10/12  The Aim is sometimes called the Charter or Opportunity Statement. It is a goal.  A written discrete, measurable, and time-bound description.  Includes the scope of an improvement effort, as viewed by the organization and the team/individual striving for improvement.  It should be developed with input from Senior Leadership to ensure support for the Team and alignment with the strategic goals of the organization.

29.  What will improve?  When will it improve?  How much will it improve?  For whom will it improve? Developed by B. Riley - Brynn.Riley@maine.gov and J. Moran – jmoran@phf.org 6/10/12

30.  When developing the AIM statement stay at a strategic level - try not to get too operational.  Determine baseline measurements while framing the AIM statement  Outline the expected tangible and intangible benefits Developed by B. Riley - Brynn.Riley@maine.gov and J. Moran – jmoran@phf.org 6/10/12

31.  AIM statements are not written once and set in cement.  A regular team task is to review the aim statement  "Remember, we’re here to increase immunizations by 50% within 12 months,"  How are we doing?  Regular review of progress quantitatively  It is acceptable to tweak the aim, as the team learns more about their work or new information changes the team’s direction. Developed by B. Riley - Brynn.Riley@maine.gov and J. Moran – jmoran@phf.org 6/10/12

32.  Section 1: Problem Description, Boundary, and Team Composition  Describe the problem or opportunity to be addressed: ◦ Example: Physician/Immunizing Stakeholders are not utilizing the Immunization Information System (IIS) tools provided, resulting in lower than acceptable immunization rates.  This process is important to work on now because of: (describe the impact this problem or opportunity is having on the agency, program, customers, employees, or the community Example: Not utilizing the provided public health improvement tools in the IIS results in improperly immunized people, wastage of State supplied vaccine, client susceptibility to disease, and community risk of disease.  Team Sponsor: _______________________  Team Leader: _________________________  Team Members: ______________________Area of Expertise: ________________ Developed by B. Riley - Brynn.Riley@maine.gov and J. Moran – jmoran@phf.org 6/10/12

33.  Section 1: Problem Description, Boundary, and Team Composition  The problem or opportunity starts with… and ends with… For Example:  The problem or opportunity starts with the lack of utilization of IIS tools* and with the assignment of staff to conceptualize and recommend an improvement plan.  …and ends with the publishing of a results document that demonstrates an increase in immunization rates via the developed methodologies and that also quantifies the cost of the improvement effort in order to identify the cost associated with sustainability  IIS* = Immunization Information System (ImmPact), Maine’s tracking systemImmunization Information System (ImmPact) Developed by B. Riley - Brynn.Riley@maine.gov and J. Moran – jmoran@phf.org 6/10/12

34. AIM Statement Sections Internal and External Tangible Benefits (cost benefit) Intangible Benefits Are benefits produced by an investment that is immediately obvious and measurable. (i.e. IT software) A tangible benefit can usually be expressed in terms of a monetary savings. Are indirect benefits which cannot be felt or touched - such as improved health access, staff morale, or the image of the health department by stakeholders. Section 2: Internal and External Benefit and Cost Description:  This opportunity has the following estimated potential benefits internally and to the external community:

35. Developed by B. Riley - Brynn.Riley@maine.gov and J. Moran – jmoran@phf.org 6/10/12 AIM Statement Sections Section 2: Internal and External Benefit and Cost Description:  What is unique to many public health organizations is the need for a way to account for intangible benefits to the community being served.  For example, tangible benefits may be a decrease in the smoking rate, an increase in the immunization rate, reduced infant mortality, etc.  An intangible benefit may be a program that contributes to increased self- esteem resulting in greater use of preventive services and adherence to an asthma management plan.

36. Developed by B. Riley - Brynn.Riley@maine.gov and J. Moran – jmoran@phf.org 6/10/12 AIM Statement Sections Internal:Tangible Benefits (cost benefit)Intangible Benefits  Reduced vaccine wastage  Increased number of doses recorded within the IIS*  Increased Immunization Rates  Increased Stakeholder awareness of the IIS*  Increased Stakeholder satisfaction with the IIS*  Increased use of IIS tools External:Tangible Benefits (cost benefit)Intangible Benefits  Increased Client Visits  Increased Services Rendered  Increased Billing Activities  Increased Immunization Rates  Additional opportunity to serve the client  Increased Community Marketing Opportunity  Improved Assessment Rates Section 2: Internal and External Benefit and Cost Description:  This opportunity has the following estimated potential benefits internally and to the external community: IIS = Immunization Information System (ImmPact), Maine’s tracking systemImmunization Information System (ImmPact)

37. Some questions to consider are:  If we implement a solution to this problem what are the potential cost categories that will be impacted and by how much?  How much will it cost to develop a solution and implement it?  What is the cost of doing nothing?  What is the cost of sustaining the solution?  Are there possible unintended consequences, and if so, what are the costs?  If we plan to use existing resources, what will we have to stop doing to divert resources to this problem/solution? Developed by B. Riley - Brynn.Riley@maine.gov and J. Moran – jmoran@phf.org 6/10/12

38.  Section 3: Current State Performance and Desired Future State  Describe the current state of the problem or opportunity and its current performance (baseline data): ◦ External Stakeholders are not using the integrated Public Health Improvement tools resident in the IIS for their use. These tools include, but are not limited to;  Immunization Coverage Reports (ICR),  Reminder/Recall Letters (R/R),  Client Vaccine Forecasting (ACIP Forecasting)  Describe the ideal future state:  What are the key driving and restraining forces impacting the current state and the ideal future state? Developed by B. Riley - Brynn.Riley@maine.gov and J. Moran – jmoran@phf.org 6/10/12

39. Section 4: Improvement Description:  This effort should improve the current state by:  Describe the improvement goals to be achieved:  The timing of these improvements:  We will measure improvements by: Developed by B. Riley - Brynn.Riley@maine.gov and J. Moran – jmoran@phf.org 6/10/12

40. Section 5: Internal and External Customer Identification:  For the following customers/clients (customers, staff or those affected by the process under improvement)  For Example:  Maine Immunization Program Staff  Consumer Services Staff  IIS Support Staff  Administrative Stuff  Physician’s Office  Nurse Staff  Administrative Support  Billing/Fiscal Support Developed by B. Riley - Brynn.Riley@maine.gov and J. Moran – jmoran@phf.org 6/10/12

41. This presentation is based on the article: Calculating the Real Value of Process Improvement: Factoring in Intangible Benefits. J. Moran and B. Riley, Process Excellent Network, April 2012 Web Address when ready: Process Excellence Network April 2012 http://www.processexcellencenetwork.com/people-performance-and-change-in-process- improveme/articles/calculating-the-real-value-of-process- improvement/&mac=SSIQ_OI_Featured_2011&utm_source=processexcellencenetwork.com&utm_m edium=email&utm_campaign=HrOptIn&utm_content=4/12/12 Developed by B. Riley - Brynn.Riley@maine.gov and J. Moran – jmoran@phf.org 6/10/12