1. Evaluation: For Whom and for What?
Centre for Social Policy Studies
University of Ghana
Daniel Doh &
Stephen Ntow

2. Outline Introduction Some historical issues
Conceptualising impact evaluation
Impact evaluation for whom?
Evaluation for what?
Final thoughts

3. Introduction
Development programming is concerned with how best to improve wellbeing of people in society.
In attempting to do this, there is the need to examine how best policies or programmes are working or not working. This brings to the fore the issue of impact evaluation
Impact evaluation serves both objectives of evaluation: lesson-learning and accountability

4. Brief history of impact evaluation
Impact evaluation emerged in the 20th century as a result of:
advances in statistics and econometrics
emergence of political and social needs
Started in medicine, education and more recently in economics and political sciences
Examples of impact evaluation methods are:
Randomised controlled trials
Difference in difference analysis
In recent times however, there is also the reliance on theory-based evaluation and other participatory (qualitative tools)

5. Source: CEGA (2010) – The Asian Foundation Impact Evaluation Workshop

6. What is common to the definitions above?
All these definitions share a number of concepts:
Social programmes: organised collective efforts to improve social welfare
Welfare outcomes: dimension of living conditions/well-being of a population (the MDGs)
Counterfactual analysis: a comparison of welfare outcomes with and without the project
Impact evaluation is a systematic process of explaining how a development intervention has or has not contributed to changes in welfare of a people

7. Shifting the Programme Design Paradigm
From
“We know what is best”…. To
“We can learn what is best in this context,
and adapt to new knowledge as needed”

8. Impact Evaluation for Whom?
The final product of an impact evaluation is like a buffet dinner served before different stakeholders and interest groups.
Each stakeholder or group goes for only what is perceived to be relevant and useful.
However, over the years some important stakeholders have not found their menu on the dinner table!

9. Evaluation for whom? Donor Community Project Beneficiaries
Governments/Other Policy Actors
CSOs
Programme Management
Project Beneficiaries
Academics

10. Evaluation for what and When?
“to recognize impact and its relevance and to use lessons learned to make the positive impacts even bigger or larger next time” (Daniel Svoboda, 2008) Impact evaluation is part of a broader agenda on evidence based policy making
When?
Early (As part of programme design)
When you introduce change.
Scale – up is envisaged

11. For Who? Government and other Policy Actors
Policies
Accountability
Donors / Development Partner
Know how their contributions affect welfare of final beneficiaries
satisfy their home country tax payers
Development practitioners and Managers
Accountability,
Tool for efficiency and effectiveness
Beneficiaries
Measurement and realization development gains.
Decision making
Civil Society
Accountability and Justification
Shape informed decisions of their constituents
Academics
Knowledge
Replication

12. Major Considerations for IE
Methodological issues:
Credible, Rigorous, uses of evidence and who?
Randomized controls or participatory methods?
Situational appropriateness: Purpose to method?
Some important considerations
Clarifying the values
Developing and/or testing a theory (Theory o Change)
Describing impact and variable
Explaining attributions
Synthesizing evidence into an overall judgment.
Contextualizing the report
Reporting findings and supporting its use.

13. Challenges & suggested solutions
Variations in implementation and environment across different sites.
Different impacts.
Absence of baselines.
Extended durations and controlling other factors.
Influence of other programs and factors
Diverse components
Resource constraints

14. Final thoughts
Impact evaluation as a professional practice and an academic exercise has found its eternal place in development practice
It is important therefore to brace ourselves with the requisite methodological skills to be able to meet demand
This methodology, as much as it adheres to conventional/standard practices must also take into account our socio-cultural context and be made meaningful to the people it affects

15. Other Definition and References

16. Rossi, Lipsey and Freeman, Evaluation
‘An evaluative study that answers questions about programme outcomes and impact on the social conditions it is intended to ameliorate’

17. Development Assistance Committee (DAC)
‘The evaluation of the ‘positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended.’

18. Development Impact Evaluation Initiative (DIME)
‘Impact evaluations compare the outcomes of a program against a counterfactual that shows what would have happened to beneficiaries without the program.’

19. Independent Evaluation Group (IEG)
‘A counterfactual analysis of the impact of an intervention on final welfare outcomes.’

20. Thank You