1. Economic evaluation in health care - II
Zahidul Quayyum
Health Economic Research Unit

2. Economic Evaluation II- Outline
Costing in Economic Evaluation
Identification of Costs, Measurement of Costs
Valuation of Costs, Main Types of Costs
Challenges in Costing
Economic evaluation in health care- Critical Appraisal
Valuing Health Care for CBA
Limitations of Economic Evaluation

3. Costing in economic evaluation
All resources used in health programmes /interventions have opportunity costs, and therefore we want to identify, measure and value resource used in health care/services, programmes and intervetions
Items to be identified for inclusion on the cost side of an economic evaluation are any resources which have an opportunity cost as a result of being used in the health care programs under consideration

4. Identification of costs
Direct costs:
Health care resources: staffing, consumables, overheads, capital
Other related services: community services; ambulance services; voluntary services
Costs incurred by patients and their families
inputs to treatments
out-of-pocket expenses

5. Identification of costs
Indirect Costs:
Time lost from work (production effects)
Production effects arise from working in the labour force and housework
Indirect costs arise because health care may result in lost production
Value of indirect cost is the value of the lost production

6. How far and wide should we go?
Consider the viewpoint or perspective of the study
Societal, NHS/health care sector, patient or employer
Identify common costs
can include only those costs that differ between the programmes
Time frame
Magnitude or quantitative importance of costs

7. Measurement of costs
Measuring resource use in naturally occurring units
Health care resources Measured how?
staffing time (hours, minutes)
consumables units/amounts consumed
overheads units/amounts consumed
capital units/amounts consumed
How?  Medical records and patient surveys
Important to report quantity of resource use
Variations between countries/regions

8. Valuation of costs Many elements are straight forward
For example staff costs - wage rates (grades) + national insurance and superannuation
Others are more difficult
e.g. overheads, capital
Calculating costs (multiply quantity of resource use by unit cost)

9. Main types of cost Total cost= sum of all the costs of producing a
particular quantity of output
Average cost = total cost /quantity e.g. cost per patient,
cost per diem
Marginal cost = the extra cost of producing one extra
unit of output
Fixed costs = cost which do not vary with the quantity of
output in the short run e.g. rent,
equipment, lease payments, some wages
and salaries
Variable costs = costs which vary with the level of output,
e.g. food, medical/surgical supplies

10. Measure and Valuation of Costs: Cost Analysis
Perspectives is important, Top down & bottom up or ingredients approach
Direct and Indirect Costs
Recurrent and Capital Costs
Overhead and Shared Costs
Apportionment and allocation, period of costing and adjustments for inflation

11. Measure and Valuation of Costs: Cost Analysis
Perspectives needs to be defined: Cost included depend on the point of view:
Health service or institution (direct costs)
Government budget (direct and some indirect costs)
Community (direct and indirect costs)
Economic Costing vs. Financial Costing
Full Economic Costing (Opportunity Costs) vs Accounting/Financial Costing- We need to show the difference, societal perspective

12. Measure and Valuation of Costs: Cost Analysis
Top down approach – Macro level costing
Identify activities, identify the cost centres, e.g. maternity ward, the related department, the overheads
Major resources used in cost centres
Measure (Quantify) and value them
Allocate and estimate costs- directly through accounting or determining allocation factor
Bottom Up Approach – Micro level costing
Identify Activities, Identify all resources and inputs, assured measure of allocation, Measure (Quantify), using appropriate allocation factor
Value them,
Top Down or Bottom up -depend on what you want/can afford to do, and how useful

13. UK costing data http://www.pssru.ac.uk/pdf/uc/uc2009/uc2009.pdf
Curtis (2009) Unit costs of health and social care, PSSRU, Canterbury.
ISD Scotland (2009) Scottish health service costs. Year ended 31st March, 2009, ISD Scotland, Edinburgh.
NHS reference costs –

14. Challenges, refinements and pitfalls
Average versus marginal costs
Double counting
Counting costs in a base year
Discounting
Dealing with overheads and capital items
Value of time

15. 1. Average versus marginal costs
What is the cost of delivery at a Scottish maternity unit? (Drummond and Mooney,1983)
Average cost = total cost of the maternity unit/number of deliveries = £540
Marginal cost = if we wanted to increase the number of deliveries in a Scottish specialist maternity units,what would be the extra health service unit cost per delivery = £510

16. 2. Double Counting Double counting including the same cost more than
once:
e.g. including time costs and fees and adding them together would be double counting, often GP fee includes time spent on activities
e.g. detailed costing of staff in an operating theatre then adding hourly cost of theatre time which already includes staffing costs as well as amounts for supplies, drugs, equipment etc.

17. 3. Counting costs in a base year
Remove the effect of inflation
Transform costs into a single base year
Inflation indices – hospital and community health services
Assuming 5% inflation:
£100 this year  £100 in a year’s time
£100 this year = £105 in a year’s time

18. 4. Discounting
Question: “Should costs (and benefits) occurring at different points in time, be given equal weighting?
If I offered to give you £1000 today OR £1000 in 5 years which would you choose?
Would you rather pay me £1000 today OR £1000 in 5 years?

19. Discounting
Generally individuals prefer to receive a benefit today and to incur a cost later
Economists call this the notion of time preference
This is important for economic evaluation as cost and benefits often arise at different points in time (e.g. prevention versus cure)
Discounting is a process by which costs and benefits which occur at different times are made comparable by expressing future costs and benefits in terms of their present value

20. 5. Overheads
Resources shared by more than one programme, department e.g. heat, light, laundry, cleaning, administration
How much should be allocated to the intervention or programme being evaluated?
Consider which costs would change if a services/programme were introduced
Number of methods
direct allocation method eg based on floor space and patient throughput

21. 5. Capital items Examples - land, buildings and equipment
The cost of capital occurs at a single point in time
Although there is one initial outlay, the opportunity cost is spread over time
Calculate the equivalent annual cost (EAC)

22. 6. Value of time Depends on the alternative use of time
Paid employment human capital approach (average wage rate)
Unpaid labour?
Dept Transport (1989) suggest to value it at 54% of average wage rate
Leisure time?
Dept of Transport (1989) suggest to value it at 43% of average wage rate

23. Economic evaluation in health care- Critical Appraisal
Drummond “10 points” check list *
Break
Exercise: Appraise and economic evaluation
Discussion and considerations about other possible types of studies
* Drummond M.F., Sculpher M.J., Torrance G.W., O’Brien B.J., Stoddart G.L.: Methods for Economic Evaluation of Health Care Programmes. 3rd Ed. Oxford University Press. 2005

24. Introduction Why critically appraise? trust the results of a study?
can apply the study results elsewhere?

25. Drummond “10 points” check list
Well-defined question?
Comprehensive description of alternatives?
Effectiveness established?
All relevant cost and consequences?
Appropriate measurement?
Credible valuation?
Differential timing?
Incremental costs and consequences?
Allowance made for uncertainty?
Appropriate interpretation of results?

26. 1. Was a well-defined question posed in an answerable form?
Examples of NOT good EE questions:
Is a chronic home care programme worth it?
Will a community hypertension screening programme do any good?
How much does it cost to run our intensive care unit?
What are the costs and outcomes of adolescent counselling by social workers?

27. 1. Was a well-defined question posed in an answerable form?
1.1 Did the study examine both cost and effects of the service(s) or programme(s)?
1.2 Did the study involve a comparison of alternatives?
1.3 Was a viewpoint of the analysis stated and was the study placed in any particular decision making context?

28. 1.3 Was a viewpoint of the analysis stated and was the study placed in any particular decision making context?
Viewpoints:
Societal; Patient; Single provider (health service, hospital, clinic, etc.); Insurer (third party payer); healthcare system; decision-maker (e.g. the Government).
Settings:
community, primary care, secondary care, tertiary care, institution, other.
Specific country where the EE was conducted

29. that is, can you tell who did what to whom, where, and how often?
2. Was a comprehensive description of the competing alternatives given?
that is, can you tell who did what to whom, where, and how often?
Example: Ingeborg et al. BMJ, Apr 2003;326:911

30. 2. Was a comprehensive description of the competing alternatives given?
2.1 Were any relevant alternatives omitted?
2.2 Was (should) a do-nothing alternative (be) considered?

31. 3. Was the effectiveness of the programmes or services established?
3.1 Was this done through a randomised, controlled clinical trial? If so, did the trial protocol reflect what would happen in regular practice?
3.2 Were effectiveness data collected and summarised through a systematic overview of clinical studies? If so, were the search strategy and rules for inclusion and exclusion criteria outlined?
3.3 Were observational data or assumptions used to establish effectiveness? If so, what are the potential biases in results?

32. 4. Were all the important and relevant costs and consequences for each alternative identified?
4.1 Was the range wide enough for the research question at hand?
4.2 Did it cover all relevant viewpoints? (Societal, community, patients, third-party payers. Other potentially relevant viewpoints?)
4.3 Were capital costs, as well as operating costs, included?

33. 5. Were costs and consequences measured accurately in appropriate physical units?
for example, hours of nursing time, number of physician visits, lost work-days, gained life years
5.1 Were the sources of resource utilisation described and justified?
5.2 Were any of the identified items omitted from measurement? If so, does this means that they carried no weight in the subsequent analysis?
5.3 Were there any special circumstances (for example, joint use of resources) that made measurement difficult? were these circumstances handled appropriately?

34. Example: Ingeborg et al. BMJ, Apr 2003;326:911

35. Example: Ingeborg et al. BMJ, Apr 2003;326:911

36. 6. Were costs and consequences valued credibly?
6.1 Were the sources of all values clearly identified? (Possible sources include market values, patient or client preferences and views, policy-makers’ views, and health professionals judgements).
6.2 Were market values employed for changes involving resources gained or depleted?

37. 6. Were costs and consequences valued credibly?
6.3 Where market values were absent (for example, volunteer labour), or market values did not reflect actual values (such as a clinic space donated at a reduced rate), were adjustments made to approximate market values?
6.4 Was the valuation of consequences appropriate for the question posed (that is, has the appropriate type or types of analysis –CEA, CUA, CBA- been selected)?

39. 7. Were costs and consequences adjusted for differential timing?
7.1 Were cost and consequences that occur in the future ‘discounted’ to their present values?
‘Discounting is a process by which costs and benefits which occur at different times are made comparable by expressing future costs and benefits in terms of their present value’
Need to allow for inflation adjustments also: so, express all cost in prices of a chosen year, then, discount to get everything to its Present Value.
7.2 Was any justification given for the discount rate used?
(i.e. Discount rate use by Government for their investments. NICE nowadays ask 3.5% for costs and utilities/benefits)

40. 8. Was an incremental analysis of costs and consequences of alternatives performed?
8.1 Were the additional (incremental) costs generated by one alternative over another compare to the additional effect, benefits, or utilities generated?

41. Cost Effectiveness Ratios: Incremental Cost Effectiveness Ratio:
8. Was an incremental analysis of costs and consequences of alternatives performed?
Cost
Cost Effectiveness Ratios:
Incremental Cost Effectiveness Ratio: B CB
Costs B Effects B
Costs B - Costs A Effects B - Effects A CA A
Costs A Effects A EA EB
Effectiveness

42. 9. Was allowance made for uncertainty in the estimates of costs and consequences?
Any Sensitivity Analysis?
One way; two-way; threshold; scenarios; probabilistic SA ?
Parameter uncertainty?
Structural uncertainty?

43. 9. Was allowance made for uncertainty in the estimates of costs and consequences?
9.1 If patient level data on costs or consequences were available, were appropriate statistical analyses performed?
9.2 If a sensitivity analysis was employed, was a justification provided for the ranges or distributions of values (for key study parameters), and the form of the sensitivity analysis used?
9.3 Were the conclusions of the study sensitive to the uncertainty in the results, as quantified by the statistical and/or sensitivity analysis?

44. 10 Did the presentation and discussion of study results include all issues of concern to users?
10.1 Were the conclusions of the analysis based on some overall index or ratio of costs to consequences (for example, cost-effectiveness ratio)? If so, was the index interpreted intelligently or in a mechanic fashion?
10.2 Were the results compared with those of others who have investigated the same question? If so, were the allowances made for potential differences in study methodology?

45. 10 Did the presentation and discussion of study results include all issues of concern to users?
10.3 Did the study discuss the generalizability of the results to other settings and patients/client groups?
10.4 Did the study allude to, or take account of, other important factors in the choice or decision under consideration (for example, distribution of cost and consequences, or relevant ethical issues)?
10.5 Did the study discuss issues of implementation, such as the feasibility of adopting the ‘preferred’ programme given existing financial or other constraints, and whether any freed resources could be redeployed to other worthwhile programmes?

46. Break 15
Exercise – Critical Appraisal of Economic Evaluation: 15

47. References
Drummond M, O’Brien B, Stoddart G, Torrance G. Methods of Economic Evaluation of Health Care Programmes. 2nd Ed. Oxford University Press. 1997
Ingeborg et al. Cost effectiveness of physiotherapy, manual therapy, and general practitioner care for neck pain: economic evaluation alongside a randomised controlled trial. BMJ, Apr 2003;326:911
Mowatt, G., Vale, L., Brazzelli, M., Hernández, R., Murray, A., Scott, N., et al. Systematic review of the effectiveness and cost-effectiveness, and economic evaluation, of myocardial perfusion scinitigraphy for the diagnosis and management of angina and myocardial infarction. Health Technology Assessment 2004 Vol.8 No.30.
NHS Economic Evaluation Database (NHS EED). National Health Service Centre for Reviews and Dissemination (NHS CRD). University of York. UK. (

48. Outline- Valuing Health Benefits in Monetary Terms
Introduction to willingness to pay (WTP)
Introduction to the literature
Methods to value health outcomes in monetary terms
Advantages/disadvantages
Conclusions

49. Valuing health outcomes
Measurement of outcomes is central to economics – resources deployed to where they will be of greatest benefit
How do we define and measure a health benefit?
How do we quantify benefits with precision?
How do we ensure comparability?

50. Valuing health outcomes using WTP
Approach to value non-market goods
WTP is a technique which can potentially be used to place monetary values on any aspect of health or health care
- including the value of human life
Use survey methods to present respondents with hypothetical scenarios about the programmes/ intervention
Respondents required to imagine that the market exists
Reveal the maximum they would be willing to pay (WTP)
Attempt to replace missing market

51. WTP Method
WTP is a method developed to provide monetary valuation of benefits or outcomes and can be used in cost-benefit-analysis (CBA).
Respondents can be asked for:
Use value (value to current patients)
Option value (availability should care be needed in future)
Existence value (Value of any concern that the programme will be available for others)

52. How do we ask WTP questions to ensure:
Acceptability
Plausibility
Consistency
External and internal validity
Avoidance of bias (hypothetical,…...)
Avoidance of embedding (part-whole bias) and ‘warm glow’ yea saying etc……

53. Methods of obtaining WTP valuations
There are two broad approaches to eliciting WTP values. One method captures values with direct questions (often called an open-ended format). An alternative method obtains values using binary choices (often called a closed-ended format).

54. Techniques for eliciting WTP
Open ended
Payment card
Take it or leave it (TIOLI)
Dichotomous choice (single, double, triple)
Bidding game
Structured haggling (exclusively used in developing countries)

55. Open ended example We are interested in the value you place on X
One way of doing this is to ask how much you would theoretically be WTP for X
What is the maximum amount of money you would be WTP for X? ……………

56. Payment card/scale
Put a * next to the amounts that you are sure you would pay 5 10 15
Put an X next to all the amounts
That you are sure you would not pay 20 25 30 35
Put a circle around the maximum amount you would be prepared to pay 50
100

57. TIOLI/DC example Would you pay £10 for X? Y/N
DC: There would be one or more follow up questions

58. Bidding game example WTP 5000? Yes No WTP 5500? WTP 4500? WTP 6000?
Stop
WTP 4000?
WTP 6500?
WTP 3500?
WTP 7000?
WTP 3000?
How much are you WTP?

59. Advantages/disadvantages of using WTP
In tune with economic theory (welfare economics)
WTP feeds consumer (patient or potential patient) preferences into decisions about health care (decision making is more democratic?)
Value more than ‘health’ and ‘time’
Acceptability, Validity and reliability of method
Biases
WTP and distributional issues
especially if WTP results are heavily influenced by ability to pay

60. Advantages/disadvantages of using WTP
If WTP is correlated with ability to pay, preferences of rich people are given greater weight
However, even if WTP values rise with income, it is not necessarily a problem
Good practice in WTP studies is to conduct diagnostic tests to see whether preferences depend on income
If there are systematic differences, one option may be to use distributional weights

61. Advantages/disadvantages of using WTP
Weighting may be applied on grounds other than income
for example, location, age, gender, patient status (current patient, at risk, not at risk)
Many potential methodological problems of WTP, some fairly easily avoided, others not
Protest votes can be a problem but the most obvious potential protest vote – respondents believe they should not have to pay for health or health care – is fairly easily avoided

62. Advantages/disadvantages of using WTP
Respondents are not (usually) asked what they are WTP in personal, out of pocket payments
The alternatives are voluntary donations but particularly (the respondent’s share of) earmarked taxation
WTP studies face a choice of trying to gain information through either open ended payment responses or the provision of cost information or payment scales

63. Advantages/disadvantages of using WTP
Failure to give respondents any payment structure invites large non response rates or non credible responses
However, supply of cost information or use of payment scales can lead to anchoring effects; in particular, WTP responses tend to cluster round the cost figure(s) or the midpoint of payment scale

64. Concluding comments Future for monetary valuation Experimental method
- different elicitation techniques
- issues of validity
Not necessarily a rival for other methods (Clinical measures, QALYs, DALYs)

65. Limitations of Economic Evaluation
Economic Evaluation techniques, needs to be carefully conducted and cautiously interpreted for a valuable aid to decision making about allocation of health care resources
Placing monetary values on health and health care, which is necessary if a full blown economic evaluation is to be conducted, is especially problematic
- in methodological terms
- more importantly, as to the assumptions made about the nature of economic behaviour towards health and health care

66. Limitations of Economic Evaluation
Increasingly, evaluation results are treated as accurate, reliable measures of people’s preferences and the process of evaluation has become much more mechanical
Actual use of Economic Evaluation is quite limited in relation to potentials
Not possible to undertake economic evaluation for all decisions due to time limitations
Very little known about decisions-makers attitudes to economic evaluation- future research