1. Evaluating Pay for Performance in Health Provincial Maternal -Child Health Investment Project in Argentina Sebastian Martinez HDNVP Presentation joint with Plan Nacer Technical Division

2. Health in Argentina Spike in Maternal and Infant mortality and morbidity following 2001 crisis, mostly amongst poor/uninsured Health coverage (2008)  Social Security (52%)  Private (10%)  Public Sector Budget (38%): uninsured Evolution of infant mortality rates in Argentina Expressed as a rate per 1,000 live births

3. Plan Nacer Health Insurance plan for:  Uninsured pregnant women (up to 45 days after birth)  Uninsured Children under 6  Target coverage of 2M women and children New strategy for health services provision:  Payments tied to performance  About 6% of health expenditures (2006 - NOA/NEA) Objectives:  Strengthen primary healthcare network  Increase accessibility and improve quality of health services  Reduce maternal and child mortality & morbidity

4. Plan Nacer’s Payment Mechanism National Level Provincial Level Target Population Providers Result Based-Financing (RBF) Fee for Service Additional Budget: Staff incentives, equipment, etc 60% Enrollment 40% Health Outcomes (Tracers) Change in health outcomes

5. Plan Nacer “Tracers” for performance payments

6. Medical Services

7. Use of Additional Budgets

8. Impact Evaluation of Plan Nacer Background: BNPP funding – critical for evaluation support early on SIEF funding secured for follow-up support TFs fund technical support Program funds data collection Evaluation team composed of MOH Plan Nacer technical area, WB staff (TTL, HDN, consultants) and survey firm

9. Impact Evaluation Questions 1. Beneficiary level: What is the impact of PN on health outcomes of pregnant women and young children? Randomized promotion design 2. Facility Level: What is the impact of PN on the provision of health services (quality of care, unintended consequences, etc)? Matched dif-dif Provincial level randomized experiments (pilots; TBD) 3. Provincial level: What is the impact of PN on coverage and utilization? dif-dif Monitoring

10. Evaluation Strategies 2006 - First Try:  Double difference based on matched sample from non-treatment areas along geographic discontinuity (NOA/NEA and neighboring provinces) 2007 – National Scale up Announced:  Need for new evaluation strategy  Randomized Promotion Instrumental Variables strategy

11. 2005200620072008Dec 2008 IE Team … P.NACER+WB+CEOP:1 st STRATEGY Jun 05 Baseline 1 st Phase Oct 06Feb 07Feb 06Oct 07 New Strategy to measure causality: Instrumental variables – Pilot Survey Baseline 2 nd Phase I.E. Provinces start to Enroll Jun 05 All Provinces are included - Provinces start to Enroll Jun 07 Provinces start Payment Mechanism Payment mechanisms start in 2 nd Phase Evaluation Timeline Plan Nacer

12. National coverage over two phases Phase 1 (APL1):  NOA, NEA  Starting 2005 Phase 2 (APL2):  Starting 2007 NOA 274.253 Sept ´08 359.189 Target NEA 249.655 Sept ´08 307.455 Target CUYO 73.974 Sept ´08 188.660 Target CENTRO 269.657 Sept ´08 1.053.288 Target PATAGONIA 29.680 Sept ´08 76.035 Target

13. Sample Structure National Sample – representative in ALL provinces  Serves monitoring and IE purposes 60 clinics per province  Sample of doctors and patients 10 households per catchment area

14. Survey

15. Snapshot of target population – preliminary baseline results from APL1 VariableMean (SD) HH Size 5.69 (2.66) Total monthly Income (pesos) 490 (372) Read/write 0.95 (0.22) Number of children 2.77 (2.09) Prenatal care received in last pregnancy 0.99 (0.12) Prenatal care before week 12 0.83 (0.38) Satisfaction prenatal care (scale 1-10) 8.29 (1.80)

16. Randomized Promotion Promote Plan Nacer through direct consumer contact:  To eligible mothers in the household (evaluation sample)  To pregnant women and health facilities Promotion in a random sample of households and localities (2/3 sample). If effective, we have a valid Instrument to correct selection bias:  Correlated with program enrollment  Uncorrelated with unobservables (randomization)  Should only affect health through the Plan (unless information is healthy) Large pilot to test promotion strategies

17. Next Steps Provider (clinic) level experiments to estimate impact of alternative program designs  Test priority questions for National and Provincial MOHs: i.e. variation in use of performance incentives  Identified first set of 4 provinces (260 health centers) to partner on provider experiments  Design experiments with provinces in early 2009 Follow-up surveys:  Mid-term early 2010 (first impact results late 2010)  Endline early 2012 (final impact results late 2012)

18. Lessons learned National programs with universal eligibility can be evaluated! Close collaboration with client key to success Plans change………  collect a baseline  have a backup strategy  Even if design is compromised, leverage data to experiment with operational questions

19. Thank You