1. Health system metrics Introduction Glion September 28-29 2006 www.healthmetricsnetwork.org

2. Demand for health systems monitoring Global Health Partnerships and other donors: enhance accountability and monitor progress of investments in health systems – Global Fund, GAVI, Child Survival Countdown 2015 etc. Universal Access to health services: HIV/AIDS, reproductive health, drugs (MDGs, G8 etc.) Assess / monitor health system performance Communication: Make health systems tangible by focusing on actionable (measurable) items –"The commitment to invest in health systems as a means of scaling up is unprecedented. It will not last unless it is possible to show results. We need to be able to show what works and why." Demand Supply Goals Framework Objectives Dashboard Performance

3. Supply side of health systems monitoring WHO /World Bank: 2004 meeting; World Health Statistics 2005 (9 indicators) and World Health Report annexes (financing) –Financing, HR, information, service delivery –Governance High and upper middle income countries: OECD quality of care, US National scorecard on health systems performance, Mexico effective coverage work Global Health Partnerships (GAVI, GFATM), WHO departments: developing shortlist of indicators for (suggested) reporting Demand Supply Goals Framework Objectives Dashboard Performance

4. GAVI and Global Fund Health System Strengthening monitoring GAVI Process indicators (short term) –N of people trained, N of vehicles purchased, etc. Output and capacity indicators (medium term) –N of facilities with trained health workers in place, % of districts with vehicles operational etc. Outcome indicators (long term) –District level DPT 3 coverage (equity indicator) –Measles coverage, –Under 5 mortality Global Fund Service delivery: 4 output (facilities with specific services), 3 outcome indicators (% of population covered by GF interventions) Human resources: 4 output (HW density, training output), 2 outcome (patient satisfaction) Community systems strengthening: 4 output (training community workers), 1 outcome (availability basic package community services) Information system & operational research: 4 output (reporting facilities), 4 outcome (surveys, HIV prevalence) Infrastructure: 2 output (health facilities with specific arrangements), 1 outcome (geographic access) Procurement and supply management: 3 output (trained technicians, stock out days TB drugs), 1 outcome (facility supply status)

5. Health system metrics - goals Develop a common strategy to monitor health systems in countries that: 1.Guides and advocates for investment in a data generation strategy to provide accurate statistics for health system indicators 2.Includes a core set of health system indicators, if possible with baselines and targets or thresholds 3.Promotes the incorporation of health system monitoring in health information systems and planning cycles (e.g. PRSP, annual health sector reviews, strategic planning) Demand Supply Goals Framework Objectives Dashboard Performance

6. Basic framework for health system metrics (2) INPUTS & PROCESSES Governance & leadership Finances Human resources Infrastructure Procurement Logistics & supplies Information INPUTS & PROCESSES Governance & leadership Finances Human resources Infrastructure Procurement Logistics & supplies Information OUTPUTS Services (availability, affordability, quality) OUTPUTS Services (availability, affordability, quality) UTILIZATION Coverage Behavioural change UTILIZATION Coverage Behavioural change OUTCOME Improved health Reduced mortality OUTCOME Improved health Reduced mortality OTHER DETERMINANTS OF HEALTH (ECONOMIC, SOCIAL, POLITICAL) OTHER DETERMINANTS OF HEALTH (ECONOMIC, SOCIAL, POLITICAL)

7. Expanding the contents of health system metrics to measure performance Areas of focus Governance: defining sector strategies, clarifying roles, managing competing demands Financing: ensuring fair and sustainable financing Human resources: having a sufficient and productive workforce Information and knowledge: ensuring the generation and use of information Service provision: ensuring adequate drugs, equipment, infrastructure Coverage: use of service by those in need Universal concerns Equity: to protect and improve health, as equitably as is possible. Being responsive to users Financial protection: to ensure people are able to avoid impoverishing health expenditures Health service quality and safety: more equitable access and use of services, ensure that effective interventions are used; services of adequate quality and safe Efficiency : to ensure that resources are used 'wisely' and not wasted

8. WHO Health System Performance Framework Adapting it for Health System Metrics Stewardship defining sector strategies, clarifying roles, managing competing demands Stewardship defining sector strategies, clarifying roles, managing competing demands Health outcomes Health outcomes Coverage reaching those who need it Coverage reaching those who need it FUNCTIONS Service provision ensuring adequate drugs, equipment, infrastructure improving organization, management and quality of services Service provision ensuring adequate drugs, equipment, infrastructure improving organization, management and quality of services Financing ensuring fair and sustainable financing, with financial protection Financing ensuring fair and sustainable financing, with financial protection Quality & safety services of adequate quality and safe harmful practices are reduced Quality & safety services of adequate quality and safe harmful practices are reduced Efficiency ensure that resources are used 'wisely' Interventions that are relevant Efficiency ensure that resources are used 'wisely' Interventions that are relevant GOALS OF THE SYSTEM Financial protection ensure people are able to avoid impoverishing health expenditures. Financial protection ensure people are able to avoid impoverishing health expenditures. Responsiveness by treating people with dignity, and ensuring confidentiality, irrespective of who they are Responsiveness by treating people with dignity, and ensuring confidentiality, irrespective of who they are Human resources having a sufficient and productive workforce Human resources having a sufficient and productive workforce Information ensuring the generation and use of Information and knowledge Information ensuring the generation and use of Information and knowledge Equitable distribution

9. Glion Meeting objectives Develop a dashboard to monitor health systems –Define the contents –Advise on reporting mechanisms Take stock of progress in health systems performance assessment –New methods –Discuss priority areas for further work, including linking dashboard to HSPA Identify priority areas for data collection and analysis investments at the country level –Clear message to international partners Demand Supply Goals Framework Objectives Dashboard Performance

10. Dashboard for health systems monitoring: focus Simplicity > comprehensiveness Parsimonious subset of indicators > full range of M&E indicators Measurability but not shying away from promoting new investments Comparable data over time and between countries National focus but include statistics within the country Low and lower middle income country focus Demand Supply Goals Framework Objectives Dashboard Performance M&E indicators Dashboard

11. Financing current situation General agreement around indicators: financing of health services (revenue collection) and financial protection WHO – annual production of statistics Data availability inadequate –70 countries ever done a full National Health Account exercise –Very few countries have regular surveys to monitor financial protection No thresholds or benchmarks for most indicators

12. Financing Dashboard example Rank*ValueYearIndicator 76$292003 Total health expenditure per capita (int $ rate) 954.3%2003Total expenditure on health as % of GDP 3612.7%2003 General gov expenditure on health as % of total general gov expenditure

13. Health workforce Current situation Some consensus on indicators, but problems with worker classifications affecting comparability Poor quality of country data as reflected by the WHO HR database Lack of consensus about data collection strategies and reconciliation of data sources –Professional data bases –Administrative data public sector –Population census –Facility census Distribution of health workforce within country: equity issues

14. Health workforce Dashboard STOCK Active health workers per 1,000 population 2.28 /1,000 (WHR 2006) ENTRY TRAINING Annual output per 1,000 population EXIT Indirect Census; migration Records; Mortality WITHIN COUNTRY DISTRIBUTION Urban – rural ratio Upper – lower quintile

15. Information Current situation Coverage civil registration systems: percent of deaths covered – limited picture and lack of change over time HMN assessment and scoring tool for all components of health information system – comparability issues Scoring of availability and quality of health statistics Scoring of data sources

16. Health Information System score Dashboard example 32% 46% 13% 36% 25% 42%

17. Governance / stewardship Current situation Multiple dimensions, some are difficult to measure: strategic vision, rule of law, participation etc. Country Policy and Institutional Assessment (CPIA): –20 indicators, instrument to guide IDA funds allocation –Criticisms: methodologically weak Not conducted in transparent manner –Potential for health sector: Lift relevant components out of CPIA Develop separate methodology and implementation

18. Service provision Current situation Includes technology and infrastructure – focus on peripheral point of service provision Many dimensions – availability, quality, safety, quality, effectiveness, efficiency => use of a score card Developed countries: quality indicators, score card in the USA; score card in Afghanistan Focus on basic services – e.g. availability of essential drugs, basic equipment –Facility census –Facility survey

19. Service provision Dashboard potential?

20. Coverage of interventions Current situation Immediate output of health systems: preventive and curative interventions; non-personal interventions Combining interventions into one index to assess health system status and performance –Co-coverage: individual data on preventive interventions for children 1-4 years (Victora et al, 2005) – requires survey data –Effective coverage: aggregate measures based on range of indicators – subnational analysis Mexico (Lozano et al. 2006) Explore possibility to combine current data on coverage into comparable indicator Investments in data collection to obtain more frequent quality coverage data

21. Co-coverage score Dashboard potential? >80%70-80%60-70%50-60%40-50%Below 40% MexicoEl SalvadorGuatemalaGuinea BissauSierra LeoneChad MongoliaColombiaBeninBangladeshTogoAfghanistan IranTurkmenistanIndonesiaKenyaBurundiNigeria SlovakiaUzbekistanZimbabweGabonCambodiaSomalia DR KoreaGeorgiaGuyanaZambiaYemenNiger Sri LankaNicaraguaParaguayBhutanPakistanCAR ThailandHondurasNamibiaGhanaBurkina FasoLao PanamaEgyptTurkeySwazilandMozambiqueEritrea JordanBotswanaBoliviaTajikistanEq GuineaTimor-Leste TunisiaMoroccoMyanmarDR CongoRwandaHaiti OmanRussiaAzerbaijanDjiboutiMauritaniaLiberia AlgeriaJamaicaPhilippinesIndiaCote d'IvoireGuinea LebanonChinaSyriaSenegalAngolaMali BosniaSerbiaRomaniaTanzaniaUganda UruguayDominican RepMalawiMadagascar CubaSouth AfricaLesothoNepal ChilePeruIraq BulgariaArmeniaGambia Costa Rica Qatar Venezuela

22. Dashboard Mortality trends Burden of disease Overall performance (Life expectancy by GDP per capita) Coverage trends (index) Service provision status and trends (indexes) Financing HR Information Governance Distribution with country (equity)

23. Stewardship: policy & governance monitoring – special data collection HR: census, labour force surveys, administrative records Financing: household surveys, administrative records (budget, exp.), NHA, PER Information: draws upon all data sources Infrastructure / technology / services: service records, administrative records Emphasis on investment in data collection

24. Health systems performance assessment World Health Survey –71 countries, results on coverage, responsiveness, input into efficiency analysis –Malaysia example Subnational assessments –District health barometer: South Africa –Efficiency analysis: Indonesia Country practices in HSPA Issues and way forward Demand Supply Goals Framework Objectives Dashboard Performance

25. www.healthmetricsnetwork.org