1. Measuring Health Systems Performance and NHA: Agenda for Health Services Research and Evaluation Measuring Health Systems Performance and NHA: Agenda for Health Services Research and Evaluation Akiko Maeda, Ph.D. Lead Health Specialist Europe and Central Asia Region The World Bank

2. Introduction NHA role in measuring health system performance NHA role in measuring health system performance Challenges in measuring efficiency and quality of health care Challenges in measuring efficiency and quality of health care Need for improved data collection and evaluation tools Need for improved data collection and evaluation tools Agenda for health services research & evaluation & NHA in developing countries Agenda for health services research & evaluation & NHA in developing countries

3. Health Sector Resource Allocation & Utilization Questions How large is the sector? How large is the sector? What are its constituent parts? What are its constituent parts? What items currently dominates in resource allocation? What items currently dominates in resource allocation? Who pays for what, and what services do they get for their payment? Who pays for what, and what services do they get for their payment?

4. Measuring Health Systems Performance Revenues /Inputs Redistribution (progressive?) Administrative Efficiency Risk-pooling/ Insurance Health Services Throughputs Allocative Efficiency Microecon. efficiency Efficacy/ Effectiveness Health Outcomes Aggregate Disease specific Socio- economic factors NHA defines financial flows from sources to service outputs

5. NHA Flow of Funds Analysis Uses of Funds: - By Institutions - By Functions Financial Intermediaries ICHA- HC ICHA- HP ICHA- HF SHA Coding Sources of Funds

6. Why Health Accounts? Why Health Accounts? Health accounts offer : Health accounts offer : –Consistent and comprehensive definition of health system taxonomy and boundaries, for comparability –Classification of resource inputs and throughputs, by functional and organizational categories –A method for costing (valuation of) health “transactions” for comparability, normative evaluation

7. Measuring Health System Performance Outcome/Inputs = Cost-benefit, cost effectiveness analysis, e.g., reduction in disease incidence rate per $$ Outcome/Inputs = Cost-benefit, cost effectiveness analysis, e.g., reduction in disease incidence rate per $$ Outputs/Inputs – Efficiency measure based on intermediate process indicators, e.g., cost per hospitalization case Outputs/Inputs – Efficiency measure based on intermediate process indicators, e.g., cost per hospitalization case Outcome/Outputs = Effectiveness/ efficacy of intervention, e.g., immunization rate and reduction in disease incidence. Outcome/Outputs = Effectiveness/ efficacy of intervention, e.g., immunization rate and reduction in disease incidence.

8. Challenges for improving the relevance of NHA NHA offers a consistent framework for measurement, but does not offer normative measures NHA offers a consistent framework for measurement, but does not offer normative measures Difficult to evaluate performance based only on aggregate expenditure data Difficult to evaluate performance based only on aggregate expenditure data Need complementary indicators of efficiency and quality: appropriateness of care, productivity measures, and other benchmarks Need complementary indicators of efficiency and quality: appropriateness of care, productivity measures, and other benchmarks Poor quality and incomplete data at micro-level (provider) limits usefulness of aggregate data Poor quality and incomplete data at micro-level (provider) limits usefulness of aggregate data

9. Measuring “Efficiency” of Health System: Challenges Major gap between aggregate macro-level data....And microeconomic performance data at provider and population level at provider and population level GAP

10. Challenges in Comparing Health System Performance Variability in the organization of health care delivery system Variability in the organization of health care delivery system Dynamically evolving technology: e.g., increasing use of day surgeries, decreasing acute care hospitalization days Dynamically evolving technology: e.g., increasing use of day surgeries, decreasing acute care hospitalization days Internal variability in performance among providers and outcome among population groups Internal variability in performance among providers and outcome among population groups Confounding factors: complex interactions between socio-economic factors and health outcomes Confounding factors: complex interactions between socio-economic factors and health outcomes

11. Example: Using NHA to evaluate Estimating Allocative Efficiency NHA functional categories: e.g., public health programs, acute vs. chronic inpatient care provides common framework for defining products, but needs to be adjusted for: NHA functional categories: e.g., public health programs, acute vs. chronic inpatient care provides common framework for defining products, but needs to be adjusted for: –Intensity, quality of care –Population – demographic profile –Evidence of clinical efficacy, cost- effectiveness

12. What can we tell from these aggregate expenditure data...?

13. Developing data collection and research capacity in the following areas: Developing data collection and research capacity in the following areas: –Household consumption and expenditure surveys, with improved designs on medical services purchased directly or through insurance –Health care provider utilization surveys –Improved quality of expenditure data at provider level (case-mix, medical procedures) Agenda for Research

14. Review of classification of services Review of classification of services –Need to review, adapt OECD SHA functional and provider classifications to suit developing country health systems Health services research agenda: Health services research agenda: –Developing affordable instruments for collecting data at provider (private & public) and population groups Agenda for NHA in Developing Countries

15. Examples of Priority Topics for Health System Research Rational Use of Drugs Rational Use of Drugs Hospitalization – tracking changes in acute and chronic care admission rates, length of stay Hospitalization – tracking changes in acute and chronic care admission rates, length of stay Administrative efficiency Administrative efficiency Public health programs – allocation on prevention and population-based programs versus personal/ clinical interventions Public health programs – allocation on prevention and population-based programs versus personal/ clinical interventions

16. Agenda for Health System Research Recommendations: Recommendations: –Focus greater attention on capacity building on health services research and evaluation –Introduce National Health Accounts within the context of health services evaluation, not as a stand-alone instrument –With improved quality of data, it should become more feasible to undertake meaningful national and international comparisons

17. Agenda for Health System Research World Bank’s standard economic and sector work focuses on Public Expenditure Reviews (PER) and Household Surveys (e.g., Living Standards Measurement Surveys) World Bank’s standard economic and sector work focuses on Public Expenditure Reviews (PER) and Household Surveys (e.g., Living Standards Measurement Surveys) Complementary capacity building should focus on supporting provider surveys, operational research on services Complementary capacity building should focus on supporting provider surveys, operational research on services

18. Measuring Health System Performance – Instruments Provider Surveys Household Surveys Administrative Data Public Expenditure Reviews Health Systems Performance Evaluation Quality & utilization reviews, technology assessment, demo. & epid. analyses

19. Health System Performance Evaluation - Low-income countries Low-income countries –Administrative data may be of limited value –Limited data on private sector activities –May need additional surveys to evaluate performance Middle Income Countries Middle Income Countries –Improved capacity to collect performance- based data through regular operational and regulatory systems

20. Health System Evaluation Process Utilization and Quality Reviews Analysis of Outputs, outcomes and Expenditures Provider level Population groups Clinical and epidemiological outcomes NHA framework on resource flows Evidence-Based MedicineService benchmarks Cost-effectiveness Analysis

21. Contact: Akiko Maeda Address:The World Bank H7-700 1818 H. St. N.W., Washington, D.C. 20433 USA Telephone:+1 202 473 3793 Email:amaeda@worldbank.org