4Learning Objectives Why is economic evaluation needed? Be able to answer the following questions:Why is economic evaluation needed?What is economic evaluation? Are there different types?When is an economic evaluation necessary or useful?
5Learning Objectives How is an economic evaluation done? More questions…How is an economic evaluation done?Who can help me with an economic evaluation?Where can I find information about economic evaluations?
6Where This Module FitsThe green and blue sections, primarily
7Next Module: Logic Model EE helps with costs and objectives
9Why EE? Evidence-based practice requires knowing Does it work?At what cost?EE starts after efficacy has been determined.Want to promote interventions that work at a reasonable cost, i.e. that provide “good value for the money”
10ExamplesIncreasing Physical Activity: Evidence shows that the following will work (The Community Guide)Community wide campaignsSchool-based physical educationStreet-scale urban design and land use policies and practicesSocial support interventions in community settingsWhich to invest in? or a mix?
11Examples, cont.Budget cuts: A department’s budget must be cut by 5%. How to do this so as to minimize the effect on the public’s health?Grant awards: A number of applications have been received for a limited pool of grant dollars. How to select the winners?
12Need to Weigh Costs and Benefits A scale can compare apples and oranges because both are measured by weightEE provides the scale – an agreed upon way to measure health benefits and costs
14What Is EE?EE examines the costs and benefits of alternative programs to inform decisions about the allocation of scarce resources using an established set of economic tools.There are four types of EE.
15Types of Economic Evaluation Cost-benefit (CBA)Cost-utility (CUA)Cost-effectiveness (CEACost-minimizationThe number of projects that can be compared declines as we move down the list.
16EE and BusinessEE methods are closely related to several business methodsReturn on Investment, or ROIMake or buy decisionA business planInvestment decision making
17It Helps to Have a Map What is the intervention? Who is the relevant group?How long should the group be followed?What can happen at each time period?DECISION ANALYSIS can provide the map.
18Example Decision TreeDowding D and Thompson C, Using decision analysis to integrate evidenceinto decision making, Evid Based Nursing, :
19Components of Economic Evaluation in Public Health InputsHealthOutputsPublicHealthProgramCostsDirect CostsIndirect CostsAverted Treatment CostsBenefitsYOLSQALYSDollarsAlways compare a new program to some alternative.
20Components of Economic Evaluation in Public Health Costs ANew Program ABenefits AChoiceCosts BProgram BComparison programMay be new or oldCould be ‘doing nothing’YOLSQALYSDollarsDirectIndirectAverted Treatment CostsHealth of the majority of the population.Benefits B
21The Product of an EEIncremental CostsEE ratio =Incremental Benefits
22The EE RatioThe EE ratio is often called the Incremental Cost Effectiveness Ratio, or ICEREmphasizes that we are comparing 2 or more interventions
23Dimensions of Economic Evaluation TypeInputsOutputsPerspectiveCMAROICBACEACUADollarsNatural Units OrganizationDollars OrganizationDollars SocietyNatural Units SocietyUtilities Society
24Every EE is Conducted from a Particular Perspective Perspective (or viewpoint) determines whose interests are paramount in the evaluationPerspective determines what costs and consequences are consideredPerspective determines how the results are interpretedKerry
25Determining the Perspective Single provider’s practiceFQHCHealthcare systemInsurer – public (Medicaid) or private (BCBS)State agencyFederal program (CDC, Medicare)SocietyKerry
26Example: Increasing Physical Activity Community wide campaignsSuppose a campaign would cost $150,000 and yield an improvement of 1,000 quality-adjusted life yearsICER = $150/QALYSocial support interventions in community settingsSuppose this would cost $200,000 and yield 2,000 quality-adjusted life yearsICER = $100/QALY
27Some Assumptions of Economics Resources are scarce.Human wants are unlimited, and more is (almost) always preferred to less.If there is perfect competition, the market will do an efficient job of allocating scarce resources to maximize profits and satisfaction. Note: society may not consider this outcome equitable.
28Some Assumptions of Economics If the market works well, then there are only 2 parties to each trade – a buyer and a seller – and they bear all the costs and reap all the benefits of the trade.
30When Is EE Needed?EE needed if there is market failure (inefficiency).Perfect competition not present.Could occur naturally or the market could be ‘broken’.EE helpful if there are concerns about inequity.
31Inefficiency Reasons for EE When the market fails, or is broken.Complex decisions with high stakes.Little or no price information available.High information costs to find out about the product.Markets involving externalities or public goods.Externalities are spillover effects from trades.Public goods are non-excludable.
32When is EE needed? Really? Practically speaking, may be mandated or requested by funding agency.Useful when comparing options internally.Useful when justifying programs externally.
33Inequity Reasons for EE Social or government decision-making processes.If the distribution of goods and services that would emerge from the market is considered inequitable, then society can legislate, regulate, or tax/subsidize.EE measures the current or desired outcomes.
34Public Health Requires EE Provides services when the market failsWIC program, health clinicsComplex decisions, high stakes - health and lifeProvides services with external benefitsImmunization programs, sanitation, inspectionsTobacco control programs
35Public Health Requires EE Provides public goods and servicesHealth monitoring and assessmentSocial marketing and public information campaignsAccess to health and equity are often concerns
37The Product of an EEIncremental CostsEE ratio =Incremental Benefits
38Possible Outcomes of an EE Aggregate Health BenefitsQuadrant IQuadrant IVSaves money,Improves healthCosts money,Improves healthAggregate CostsSaves money,Worsens healthCosts money,Worsens healthQuadrant IIIQuadrant II
39Measuring CostsAll economic evaluation methods require listing and measurement of costs.One tricky point.Mathematically, a benefit can be a negative cost, and a cost can be a negative benefit.
40Negative Costs and Benefits Whether some items are negative costs or benefits depends on the method.Makes a difference because we are constructing a ratio.General idea: all items falling on the health system are costs, positive or negative.
41Types of Costs Direct (or program) costs Indirect costs Time and travel costs to participantsAverted productivity losses (a negative cost)?Cost of treatment during gained life expectancy ?Averted treatment costs (a negative cost)
42Direct Costs All costs of the intervention Labor Supplies Rent UtilitiesCosts of treating side effects of the intervention, if any
43Indirect Costs Also called spillover costs. Time and travel costs to participants.Costs to parties outside controlling program or agency.Averted productivity losses (a negative cost)?Cost of treatment during gained life expectancy?
44Averted Productivity Losses The present value of future wages gained.The intervention increases the worker’s longevity and/or reduces disability.Used in cost-benefit analyses.Not included in cost-utility analyses.Double counting of the benefit.
45Costs of Treatment During Gained Life Expectancy If the intervention saves a life, that person will die of something else later on. What are his/her expected total medical expenditures in the added life years? A controversial cost.Pro: these are expenses that would be avoided in the absence of the intervention, and therefore should be counted.Con: health expenses should not be singled out for inclusion.
46Averted Treatment Costs The averted costs to society of treating persons for the disease are subtracted from total costs.Included in the cost side because they impact the health care budget.A negative cost, not a benefit.
47Measuring Benefits Benefits can be measured in many ways Different units of measurementDollarsYears of life savedQuality adjusted years of life gainedSpecific health outcomeDifferent time frames for the outcomeIntermediate outcomeFinal health outcomes
48Measuring Benefits Prevention poses special problems: Benefits may be: small and cumulative.complex and interrelated.Are difficult to measure or validate without large samples of panel data observed over long time periods.Consequently, often focus on intermediate outcomes.
49Measuring Benefits Prevention poses special problems: Aim is often to change behavior among more people.Increases costs in the short term.Community versus individual approach.Economies of scale possible.Provide walking trails; tax gasoline.Individual interventions often more costly.
50How to Put it Together?Have costs and benefits, how to put them together?Depends on the type of EE.
51Definition Cost-minimization Analysis Populations served must be identicalOutputs for all alternatives must be identicalTimeframes must be the sameSearch for alternative that yields the lowest total cost
52CMA Example Deliver dental sealants to school children Choose staffing that minimizes program cost per studentNo supervisionGeneral supervisionIndirect supervisionDirect supervisionScherrer, CR, et al, Public Health Sealant Delivery Programs, Medical Decision Making, Nov-Dec 2007
53Definition Return on Investment (ROI) or Business Case Answers question: Will an activity pay for itself?Analysis of net discounted cash flow to entity paying for an interventionTypically short-term (1-5 years)Perspective is key to determining costs and consequences to include in analysisKerry
54Why develop a business case? A positive business case increases the probability that a prevention intervention will be sustainedROI analyses are most easily understood by administrators and budget officersBrownson CA and Kilpatrick KE, Building a Business Case for Diabetes SelfManagement: A Handbook for Program Managers, Robert Wood JohnsonFoundation, 2008 (available at
55Return on Investment Example CareOregon (Portland): initiated case management for high-risk Medicaid enrollees with multiple comorbiditiesStudy design: before-after design compared to baselineInvestment costs: $526,290Discounted savings: $6,423,776Return on investment: : 1Greene SB, et al, Searching for a business case for quality in Medicaid managed care, Health Care Management Review, 2008, 33(4),
56Definition Cost-benefit Analysis Cost-benefit analysis values both the costs and benefits of a program, project, or treatment in monetary terms.Result of analysisThe net benefit of the project (e.g. benefits minus costs, or $12,000) ORThe ratio of costs divided by benefits (e.g. ½).
57Cost Benefit ExampleIntervention: Neighborhood-based program to prevent teen pregnancyProgram costs: $9,386 per participant per yearEffects: reduced teen pregnancy from 94/1000 to 40/1000Cost: $26,142 per birth avertedSaved: $81,256 society costs/birth avertedRosenthal MS, et al., Economic Evaluation of a Comprehensive TeenagePregnancy Prevention Program, Am J Prev Med 2009;37(6s1)
58Example ICBR Smaller ratio is better ICB ratio = $26,142 = .322 $81,256Smaller ratio is better
59Cost-benefit Analysis Gold Standard for EE.Allows for economic comparison of widely disparate publicly funded programs in such areas as health, education, and the environment.Problem: valuing a life in monetary terms.
60Definition Cost-effectiveness Analysis Cost-effectiveness analysis measures the benefits of a program in naturally occurring health units, such as lives saved. Example of a study result: $10,000 per life saved.
61CEA Example Intervention: Smoking cessation program in the workplace Effect measured: Number of people who quit smokingICER = $596 cost per additional quitterThis cost was less than “high intensity” interventions by cliniciansTanaka H et al. Effectiveness of low-intensity intra-workplace intervention on smoking cessation in Japanese employees: a three year interventionTrial, J. Occup Health 2006; 48(3):61
62Cost-effectiveness Analysis CEA formerly most common form of EE conducted in health arena.Limited in its ability to report outcomes.Often, there are multiple outcomes.Limited in its ability to compare interventions.
63Three Projects Example A nursing program for newborns and their parents costs $50,000 per year and serves 50 infants with high-risk conditionsA screening program that visits local malls and community centers costs $15,000 and provides information to about 5,000 personsA vaccination program costs $100,000 and provides vaccinations to 20,000 area children
64Three Projects Example and CEA Screening programCost-effectiveness ratio is $15,000/5000 or $3 per attendeeNurses for infantsCost-effectiveness ratio is $50,000/50 infants or $1,000 per infantVaccination programCost-effectiveness ratio is $100,000 /20,000 or $5 per child
65Definition Cost-utility Analysis Cost-utility analysis compares the costs of different programs, projects, or treatments with their outcomes measured in “utility based units” which are related to a person’s health related quality of life. Example of a study result: $10,000 per quality-adjusted life year, or $10,000/QALY.
66Definition Cost-utility Analysis Becoming the most common form of analysis.Widely used in Britain and Canada.Allows comparison of many projects with health-related outcomes.Often called cost-effectiveness analysis; closely related.
67Quality adjusted life years CUA Formula(direct costs + indirect costs - averted treatment costs)Cost - utility ratio=Quality adjusted life yearsOutput of CUA is ‘cost per QALY’
68Cost-utility Outcomes Outputs are measured in terms of a preference-based outcome measure.Quality Adjusted Life Years (QALYS): the number of years at full health that would be valued equivalently to a given number of years of life experienced with a disease or disability.Other measures are available, this is the most common.
69Why Measure Quality of Life? Health care outcomes are multidimensional.- Length of life, or mortality.- Quality of life, or morbidity.Allows for more than one disease or health problem to be compared.Considers the individual’s preference for health outcomes.
70How QALYS Are MeasuredOne year of life in excellent health is assigned a value of 1; death is given a value of 0.A health state is described and its utility or quality elicited.The value assigned to quality of life is referred to as health utility..51
71An example of estimated health utility values for diabetes 4/2/2017An example of estimated health utility values for diabetesThis slide shows how the quality of life change with an increase number of diabetes related complications. For example, for person with diabetes treated with diet and exercise , they utility value is 0.69 and their quality of life will decrease another 0.02 if they went to on oral agent and so on. What does o.69 means? It mean that people in a perfect health would willingly give up 0.31 year or about 4 months of their life to avoid a year of living with diabetes.Coffey et al Diabetes Care71
73Problems With QALYs Subjective and difficult to measure. Whose QALYs should count? A representative sample of the population or the affected group?QALY scales will differ depending upon factors such as age, gender
74Stop Smoking ExampleAfter considering your community and its public health problems, your organization has identified smoking as a problem you can address.Two interventions are being considered; only one can be done.
75Stop Smoking Example Using CUA Multicomponent interventions that include patient telephone supportSuppose this would cost $150,000 and yield 1,000 quality-adjusted life yearsHealthcare provider reminder systemSuppose this would cost $200,000 and yield 2,000 quality-adjusted life yearsICER for B vs. A =$50,000/1000 QALYsor $50/QALY
76Cost Utility ExampleIntervention: Diabetes self-management programs in primary care settingsProgram costs: $866 per participant per yearEffects: 87.5% benefited, A1c -.5%, total cholesterol. -10%ICER: $39,563/QALY savedBrownson CA, et al., Cost-effectiveness of Diabetes Self-managementPrograms in Community Primary Care Settings, Diabetes Ed, v. 35, no.5, 2009
77How Do I Know If It’s Cost-Effective? Are we almost there yet?Once I have the ratio, how do I know if it’s too high, too low, or just right?
78Is It Worth It? Results can be used internally or externally To rank programs internallyTo argue for external supportIntermediate results, such as productivity gains, can be highlighted for some stakeholders
79Using Results Internally For internal useRank options from lowest to highest ratio.Start spending on lowest ratio, move on until the money is exhaustedWhat have we spent before?
80Using Results Externally The ICER is compared to a threshold valueSuggested U.S. threshold is $50,000 to $100,000 per QALY at minimum
82Two Other Important Features Before study is complete, should considerDiscountingSensitivity analysis
83Discounting Time value of money. A dollar in the future will be worth less than a dollar in the present.Needed to compare present value and future value of benefits from project.Recommend a discount rate of 3-5%.Discount rate chosen can affect results.
84Discounting in Practice Suppose you want to find the present value of $100 received in 10 years. Several options for finding this.Tables in finance and accounting books.Excel or other spreadsheet programs.The internet: Google “present value calculator” and several pop up.
85Sensitivity AnalysisEE is based on estimates and assumptions – want to vary them and see how robust the results are.Variables to test in sensitivity analysis should include the “top 3” or “top 5” “wobbliest” assumptions.Analysis should be redone varying the assumptions.
86EE Results ReportingThe ICER(s) should be reported for the intervention(s) studied for the base case.ICERs may be reported for subgroups of the population.ICERs should be reported for different assumptions (sensitivity analysis).
87How is This Done in Practice? Back of the envelope EEDraw a decision tree, follow the costs and benefits, form the ratioNo economist requiredPrimary data collectionOften alongside an interventionCollect cost and benefit dataEconomist works with team
88How is This Done in Practice? Synthetic EEFollow a hypothetical cohort of personsUse the literature to obtain cost and benefit estimatesRely on modeling and computer analysisEconomist leads the team
90Who Does EE? Some economists are trained to do EE. Most EEs require a multi-disciplinary team.Few public health agencies have the resources to have a staff economist – so think collaboratively – local colleges.
92Web ResourcesThe economics section of The Community Guide, it includes systematic reviews of EEs for recommended activities.The Cost Effectiveness Analysis Registry at Tufts – New England Medical Center. Includes EE results and QALY estimates for numerous conditions.
93Web ResourcesThe Cochrane library, a reliable source of information on the effects of interventions in health care. Economic evaluations are available ator by clicking from the web site above.
94Web ResourcesThe Centre for Reviews and Dissemination at the University of York. Includes several searchable databases; of particular interest is the NHS Economic Evaluation Database (NHS EED), which provides article summaries, similar to the Community Guide.The National Institute for Clinical Excellence. A more general database (the prior website is reachable through this site, for example) it includes RSS feed capabilities.
95NHS – EED Provides summaries of EE articles Easily searchable Follows a standard formatSummary is about 2 pages or lessCan then click on full report, which may be several pagesEasily searchableGood accessible instructions for searching
97Reference BooksDrummond MF, Sculpher MJ, Torrance GW, O'Brien BJ, Stoddart GL. Methods for the economic evaluation of health care programmes. Third edition. Oxford: Oxford University Press; 2005.Muennig P (Contributing Editor, Kahn K). Designing and Conducting Cost-Effectiveness Analysis in Health and Medicine. San Francisco: Jossey-Bass, (2nd edition forthcoming).
98Search TipsGeneral strategy: search for the intervention of interest AND economic evaluationOften easier to run 2 searches – 1 for intervention and 1 for economic evaluation, and then combine them
99Search TipsIf you want any type of economic evaluation, search for ‘cost-benefit’ OR ‘cost effectiveness’ OR ‘cost utility’“cost-effective” is a buzzword that will net a lot of articles!Articles in intervention journals will be more verbose on the EE and terse on the intervention; articles in economic journals will be the reverse.
100Economic Journals With an EE Focus Health EconomicsJournal of Health EconomicsCost Effectiveness and Resource AllocationHealth Technology AssessmentApplied Health Economics and Health PolicyValue in Health
101SummaryEconomic evaluation is the comparison of costs and benefits to determine the most efficient allocation of scarce resources.Economic evaluations can use existing or new information and can provide a reliable tool for decision making among public health professionals and policy makers.Though relatively sophisticated, the underlying logic and structure of an economic evaluation can be understood.
102SummarySeveral challenges (e.g., inconsistent quality, methodological issues, difficulties in implementation) should be kept in mind when considering the use of economic evaluations.Economic evaluation will be increasingly used, especially in times of limited public health resources, and practitioners must be able to understand them so that they can argue for setting appropriate public health priorities.
103“It is our choices … that show what we truly are, far more than our abilities.” J.K. Rowling, Harry Potter and The Chamber of Secrets, 1999