What do we know and don’t know… We know that high impact interventions lead to improved health outcomes .. We do not know how to best produce demand and effective coverage with these interventions in specific contexts ….what economists call the “production function”… We know the inputs.. But not the incentives, institutional and management arrangements The “missing middle” or “black box” ..

When we do not know there are consequences… Government ‘s policies are “shooting in the dark” Public money is not used efficiently.. Aid effectiveness is low.. Ideology prevails..

An example of an ideology driven debate…user fees Proponents: user fees provide incentives to providers, reduce absenteeism, improve quality, improve accountability to users/clients Opponents: user fees reduce access and utilization, and are a burden on the poor’s households budgets Evidence ?

An example of an ideology driven debate …user fees Hundreds of publications on the topic.. Yet less than 5 rigorous studies based on population surveys of service use and randomized control group Conclusions of these studies are mixed..

Uganda: Utilization of Curative Services Trending Upwards Public Expenditure Review 2008 The World Bank The World Bank

Uganda: little effect of user fees abolition on MDGs related High Impact Interventions Source: DHS 2000 and 2006

The promise of Results Based Financing.. Setting Evidence Based Policies : The Rwanda example

Rwanda is back on track to reach the MDGs Under five mortality trends with MDG target for 2015

Increase in utilization of high impact services Trends in assistance at delivery : Years 2000, 2005, 2007 Percentage (%) of women delivered by a health professional

Rwanda : Intake of Family planning tripled in three years.. One such existing user earns about $0.37. A new user earns $1.83 for the health center. The above graph shows the AVERAGE number of such services, per health center per month. This reflects 98.2% complete reporting for 2006, and 100% complete reporting for 2007. All 295 health centers in the 23 districts reported completely, and timely. 10

Performance-based Financing (PBF) Developed after extensive piloting from 2001-2005 Objectives Focus on maternal and child health as well as communicable diseases (MDGs 4 & 5) Increase quantity and quality of health services provided Increase health worker motivation Financial incentives to providers to see more patients and provide higher quality of care Operates through contracts between Government Public and Private Health facilities providing services

Evaluation Questions: Did PBF… Increase the quantity of contracted health services delivered? Improve the quality of contracted health services provided?

Researcher & Policy Maker Collaboration Research Team Paulin Basinga, National University of Rwanda Paul Gertler, UC Berkeley Jennifer Sturdy, World Bank and UC Berkeley Christel Vermeersch, World Bank Policy Counterpart Team Agnes Binagwaho, Rwanda MOH and CNLS Louis Rusa, Rwanda Rwanda MOH Claude Sekabaraga, Rwanda MOH Agnes Soucat, World Bank A collaboration between the Rwanda Ministry of Health, CNLS, and SPH, the INSP in Mexico, UC Berkeley and the World Bank

Acknowledgments Funding by: World Bank Government of Rwanda (PHRD grant) Bank-Netherlands Partnership Program (BNPP) ESRC/DFID GDN

Evaluation Design During decentralization, phased rollout at district level Identified districts without PBF in 2005 Group districts into “similar pairs” based on population density & livelihoods Randomly assign one to treatment and other to control MOH reallocated some districts to treatment With decentralization, some new districts had PBF in an area of the new district – must be treatment Unit of observation is health facility