1. David Booth Alison Evans
Evaluation Network follow-up to the Paris Declaration: An options paper
David Booth
Alison Evans

2. Overview What sort of challenge to evaluation?
What would be the value added?
What should be the focus?
Is it possible?
Tradeoffs and options
Recommended way forward
Timing and next steps

3. What sort of challenge?
One that can’t be refused – the Declaration itself calls for independent evaluation “to provide a more comprehensive understanding …”
An important one – the Declaration contains an unprecedented set of mutual commitments, leading to major efforts and investments
An exciting one – it is possible to articulate the “program logic” of the Declaration and interrogate the underlying theories of change

4. What would be the value added?
The monitoring scheme will tell us something about what is happening, so will peer reviews,
but we need to understand also how and why/why not
The monitoring has also made various pragmatic compromises to get standardised data quickly and without imposing on countries
an evaluation effort could be more balanced and wide ranging (and would be required to be so)

5. What should be the focus?
The illustrative results chain in the paper shows the conceptual scope, from inputs to impacts
There is no necessary implication that the program theory is agreed or that it is right
The principal focus would be on the top levels, and on formative evaluation questions
which does not imply results and impacts do not matter!
“Negative” findings – why things are not working, or are working for unexpected reasons – should be valued as much as “positive” findings

6. Is it possible?
So long as the formative purpose is kept firmly in mind
So long a ambitions regarding the results chain are well controlled, and the framework is used selectively
So long as it is possible to draw on a wide range of types of evidence to answer a selected sub-set of questions
These include
are we doing things right?
are we doing the right things? And
In what range of situations might these things work?
But probably not “is it cost-effective?”

7. Tradeoffs and options
Preparation of a common evaluation framework is widely supported – considered intrinsically useful
Otherwise, options involve significant tradeoffs:
between buy-in and “criticality”
between country- and cross-country learning
between low costs and adding value
No approach to design of an evaluation process will have all of the desired qualities
Needed: an optimal solution or good compromise

8. Recommended way forward
Four loosely-connected activities, offering distinct strengths
Development of a common framework – an agile, consultative but fairly top-down process
Country-led country evaluation work – based on self-selection, building closely on existing monitoring initiatives and localising elements of the common framework
Thematic studies focused on donors – similar principles
A medium/long-term programme of synthesis and meta-evaluation – triangulating diverse data to address the big questions in collaboration with MT Monitoring Plan of JV

9. Timing and next steps Considering the time-line to the HLF of 2008:
Work on the framework should happen in 2006
To be sufficiently thorough, promoting the idea of country-led evaluation also needs to start soon
Dialogue with the JV to ensure well-integrated processes is vital
If the general approach is agreed, a task team, preferably with some country representation, would be required to take the work forward

10. In summary In some significant and useful ways, it can be done
It is very important that it should be done
Appropriate, realistic expectations are needed, especially before 2008
“Walking on four legs” may look untidy, but it will do the job, recognising the constraints