1. Paying for Performance in Haiti Rena Eichler, PhD Broad Branch Associates renaeichler@comcast.net Results Based Financing to Reduce Maternal, Newborn, and Child Mortality, (Session 5; October 21, 2008)

2. Outline What motivated the introduction of payment for performance (P4P) in Haiti? Pilot Roll out: Design changes and results Lessons and challenges

3. Overview of P4P in Haiti USAID funded project (1995-present) to strengthen capacity to provide essential health services, managed by Management Sciences for Health. 2.7 million people covered by 2005 NGOs are contracted to provide a defined package of services. Technical assistance is provided to enhance institutional capacity. Beginning with a pilot in 1999, progressively more NGOs transition from being reimbursed for documented expenditures to payment for results.

4. What motivated the introduction of P4P in Haiti?  1997 Population-based survey found wide variations in performance (23 NGO service areas), which indicated improvement was possible.  Full immunization coverage ranged from 7% to 70%.  Contraceptive prevalence rates ranged from 7% to 25%.  Prenatal care ranged from 21% to 43%.  Attended delivery ranged from 53% to 87%. No correlation between performance and cost per visit: rough estimates of average costs per visit range from $1.35 to $51.93! Communities’ health needs not met Always “legitimate” excuses for non-performance Business as usual … There had to be a better way!

5. Experiment: Shift from “ payment for expenditures ” to “ payment for results ” “Before”: Expenditure-based Reimbursement Expenditures reimbursed up to a negotiated budget ceiling NGOs submitted monthly vouchers that document expenditures Routine financial verifications and financial audits Incentives were: Justify higher budgets. Accountability for spending on inputs rather than on results.

6. “After” The change: Payment for results Negotiated performance targets relative to NGOs own baseline. Puts organization at risk for a % of total yearly budget if targets are not met 5% of annual negotiated budget is “withheld” : tied to attainment of performance targets. Provides an additional bonus for strong performance Additional 5% if performance is strong. Fixed price subcontract with fixed quarterly payments plus award fee. Incentives are: Achieve results. Use inputs effectively.

7. Stakeholder consultations were essential NGOs had already signed contracts for the existing year. It was thought they might be willing to renegotiate IF new contract terms offered the possibility of being better off. First step was to consult with NGOs to obtain input into design and to generate this “buy in”. NGOs perceived to be well managed met to explore interest to participate in a pilot. NGOs managers agreed that payment for results was appealing. Consistent with their missions. Increased flexibility. Reduced reporting on expenditures. The existing context influenced the form of performance based payment that was acceptable to recipients.

8. Pilot Phase Indicators Performance Indicators 1. 10% increase in full immunization coverage 2. 20% increase in number of pregnant women receiving at least 3 prenatal care visits 3. 15% increase in the number of mothers with full knowledge of oral rehydration therapy 4. All Institutions and half of outreach points with at least 3 modern FP methods 5. 25% reduction in the discontinuation of family planning 6. 50% reduction in waiting time for child attention 7. Well defined community committees with appropriate coordination with Ministry of Health

9. Pilot Results

10. Incentives The “ push ” : Financial risk (potential to lose the “ withheld ” amount) provides incentives to achieve performance targets. The “ pull ” : Opportunity to earn the bonus provides strong incentives to achieve performance targets. P4P provides incentives to implement strategies that achieve results. Some changes include: Improved management Better use of information for management decisions More motivated staff Effective outreach to challenging populations More efficient and effective use of inputs P4P inspires innovation.

11. What is the policy relevant RBF “package”? Financial incentives- only one part. Increased flexibility in the use of funds. Technical assistance. Creation of a network that enables NGOs to learn from each other. Increased emphasis on (and use of) information to monitor results.

12. RBF is not static! Pilot phase (pilot):1999 Standard list of health indicators Performance verified by independent firm with community-based surveys Phase I: April 2000-December 2001 NGOs selected to participate based on perceived readiness Standard list of health indicators NGO self-reporting complemented by validation by an independent firm Phase II: January 2002-December 2003 NGOs selected to participate based on perceived readiness Standard list of health indicators Random selection of health indicators from an expanded list Addition of a standard list of management indicators 50-50 split between health and management indicators for payment NGO self-reporting complemented by validation by an independent firm

13. RBF is an evolving strategy Phase III: January –December 2004 NGOs selected to participate based on perceived readiness Random selection from 2 “packages” of indicators including both health and management NGO self-reporting complemented by validation by an independent firm Phase IV: January-December 2005 All NGOs in RBF. Payment conditional on benchmarks plus performance targets that were the same for all NGOs Amount at risk increased to 12%. NGO self-reporting complemented by validation by an independent firm

14. Results The following suggests that RBF is at least partially driving the better results: Big jump in performance between the year prior and the first year in P4P. Big improvement in project performance in 2005 when all NGOs were in P4P. Regressions that adjust for NGO specific effects and year effects show a significant impact of RBF on results. However, it is not possible to disentangle the impact of: Financial incentives vs. other aspects of the “RBF package” Is it the payment mechanism or other interventions (TA, increased funding, flexibility, networking)? Or a combination? Are NGOs in RBF more capable to begin with?

15. Average Performance Changes: From prior year to first year in P4P

16. Regression results Results suggest that P4P is associated with a 13 to 24 percentage point increase in immunization coverage per year. Translates to 15,000 additional children per contract period immunized because of P4P. Results suggest that P4P is associated with a 19 to 27 percentage point increase in the number of women who delivered babies with the assistance of trained attendants per year. Implies that 18,000 additional women per contract period gave birth more safely because of P4P.

17. Some lessons… P4P can serve as a catalyst to introduce significant management and organizational changes that strengthen service provider capacity. The “all or nothing” nature of performance targets encourages long term planning, innovation and system change The organization serving as “payer” must make significant changes in its structure and management systems; it must integrate its financial mgmt, contract administration, performance management and capacity building functions. Shifting from surveys to NGO reported data combined with audits to verify data accuracy is more cost effective. It creates incentives for NGOs to improve their M&E and to routinely use their own information for ongoing management “Waiting” for NGO capacity to be sufficiently enhanced before shifting them to a performance-based payment system is not necessary – instead of a pre-requisite for P4P, “capacity development” turned out to be a result of P4P.

18. Thank you