1. Building National Systems for Monitoring and Evaluation (M&E): Lessons from Africa and Other Regions
Presentation to PREM Learning Week
Workshop on Implementing Results-Oriented Poverty Reduction Strategies
Washington D.C., 22 April 2008
Keith Mackay
Independent Evaluation Group
The World Bank

2. Structure of Presentation
Five Main Uses of M&E
What Do We Mean by “M&E”?
International Case Study: Colombia
International Lessons
Some Strengths in African Countries
Some Difficulties in Africa
A Way Ahead
Useful Resource Materials

3. Five Main Uses of M&E
To support budget decision-making – i.e., performance-based budgeting
3 main types: direct, indirect, presentational
To support national, sectoral and sub-national planning
To design new policies and programs
To assist ministries, entities and sub-national governments in their management
To strengthen accountability relationships

4. What Do We Mean by M&E?
Performance indicators  inputs, processes, outputs, outcomes, impacts
Clarify program objectives, program logic / results chains
Service delivery surveys
Participatory M&E
Rapid evaluations
Rigorous impact evaluations
Cost-benefit analysis
Etc – many tools, methods, approaches
Each M&E tool has strengths, costs, limits

5. International Case Study: Colombia
M&E system managed by the Department of National Planning (DNP), with strong support from the President’s Office
On-line monitoring sub-system for monitoring and reporting government progress vis-à-vis Presidential Goals
Ambitious agenda of impact and other evaluations (∑15 underway)
DNP provides technical assistance to a few ministries/agencies to develop M&E, and to municipalities to pilot monitoring sub-system and performance-based budgeting

6. Colombia’s M&E System  Strengths
Very high utilization of monitoring sub-system by President for oversight of ministers and ministries – via performance targets – and for accountability, i.e. “social control”
Rigorous impact evaluations conducted externally, and have high credibility
Collaborative approach between DNP and sector ministries/ agencies, and with municipalities

7. Colombia’s M&E System – Weaknesses
Insufficient reliance on M&E information to support national planning and budget decision-making – this might soon change
Weak coordination of M&E roles / functions within DNP and with central and sector ministries
Monitoring sub-system data quality perceived as low
Too high reliance on donor funding for the M&E system – low level of government funding support

8. Colombia  Next Steps Gov’t to strengthen institutional framework
committee will plan, oversee evaluations and ensure recommendations implemented
Efforts to link M&E to core budget, planning work
Broader range of evaluations to be done
Less reliance on donors to fund evaluations
Support for pilot regions, municipalities to strengthen existing M&E systems
Stronger regulation of data quality; data harmonization; external data audits
World Bank project ($10m over 5 years)

9. International Lessons (i)
Demand and supply sides both need careful attention – a mistake to view M&E, and M&E systems, as mainly technical issues
On supply side, (i) caveat multiple, competing, uncoordinated systems; (ii) ensure data reliability, credibility; (iii) rationalize data collections, harmonize
Avoid common mistake of over-engineering the M&E system

10. International Lessons (ii)
Monitoring information – performance indicators – are useful but it helps to have evaluations to understand reasons why performance is good or bad
Importance of training – to be able to conduct M&E and to use M&E information

11. International Lessons (iii)
On demand side, a powerful champion of M&E plays key role – ideally centrally-driven, by capable ministry (or ministries)
Home-grown demand for M&E likely to be most sustainable (Tanzania, Uganda, South Africa)
Need incentives to ensure M&E is taken seriously – on both the supply and demand sides
Limitations of relying on government laws, decrees, regulations

12. International Lessons (iv)
Diagnosis of existing M&E – supply and demand aspects – is an important first step
Need continuous review and periodic modification of a government M&E system
via M&E of the M&E system
successful countries developed their M&E system opportunistically
building a government M&E system is long-haul effort, requiring patience
Keep supply and demand in balance – utilization of M&E is the metric of success

13. Some Strengths in African Countries
Active national and regional evaluation associations – e.g., RenSE, SAMEA, AfrEA
Emphasis on Poverty Reduction Strategies increases the focus on M&E
Shift to programmatic lending increases reliance on governments’ M&E information
Some NGOs are active in M&E
Many governments have some key staff who understand and are committed to M&E
High level of donor enthusiasm and support

14. Some Difficulties in Africa
Disconnect between planning and budgeting
Multiple, uncoordinated data systems
And multiple, unharmonized donor requirements
Under-used data (even when good quality)
Donor emphasis on national statistics agencies
Most countries have poor systems for tracking government spending
Problem of the “missing middle”
M&E often supply-driven – thin extent of country demand for M&E
Danger if M&E efforts are largely donor-driven

15. A Way Ahead (i)
Start with a diagnosis of current M&E – strengths, weaknesses, options for the future
Endeavor to build a coalition of support within government, and with donors – explain to officials “what is M&E?”, and tell them “what is in it for you?”. Thus raise awareness of, and demand for, M&E
Work with a sector ministry or agency to develop a demonstration pilot

16. A Way Ahead (ii)
Push donors to support the government’s M&E agenda, and to fund evaluations
Consider some or all of the following as possible key elements of an M&E system:
national and sectoral statistical collections – especially relating to national priorities such as the MDGs
sector ministries’ administrative data
financial management information systems – for better tracking of government spending
public expenditure tracking surveys to identify “leakage” of funds and possible corruption

17. A Way Ahead (iii) Possible key elements of an M&E system (continued):
service delivery surveys of client satisfaction and perceptions of government service quality
rapid appraisals – e.g. of “problem” projects or programs
But caveat: “less is more”, so it is important to monitor the extent to which each type of M&E information is actually used
and, where utilization is low, identify the reasons and try to address them

18. Useful Resource Materials
Keith Mackay, How to Build M&E Systems to Support Better Government, The World Bank,
Arild Hauge, The Development of Monitoring and Evaluation Capacities to Improve Government Performance in Uganda, ECD Working Paper No 10, IEG, World Bank, October /6172aa31c69bcf dc1006c9d24/$FILE/Uganda_ECD_Paper_10.pdf
World Bank, A Diagnosis of Colombia’s National M&E System, SINERGIA, ECD Working Paper No 17, IEG, World Bank, February

19. Useful Resource Materials
Keith Mackay, A Diagnostic Guide and Action Framework, ECD Working Paper No 6, IEG, World Bank, January /7f2c924e183380c fc /$FILE/ECD_WP6.pdf
Independent Evaluation Group, Monitoring & Evaluation: Some Tools, Methods and Approaches, World Bank,
Independent Evaluation Group, Influential Evaluations: evaluations that improved the performance and impacts of development programs, World Bank,
World Bank website on Building Government M&E Systems: