1. Reflections on Revising the Guidance: An Evaluation
Dr. Thania Paffenholz
Oslo 17 February 2011

2. Improved Evaluation Practice
Output
Outcome 1
Outcome 2
Impact
Improved Evaluation Practice
Awareness for/of EVAL + CPPB field
Draft Guidance
Contribution to better Quality in CPPB work
Outcome 3
Results
Input
2 DAC Networks + Experts

3. Evaluation along Criteria
Relevance
Is the Draft Guidance responding to the needs of CPPB field?
Is the Draft Guidance responding to the needs of evaluation field?
Effectiveness
Intended outcomes:
How effectively has the Draft Guidance replied to the needs of the two main target groups: Evaluators and Evaluation Managers?
Unintended outcomes:
What kinds of other outcomes (positive and negative) have the Draft Guidance so far produced?
Sustainability
How can the Guidance be used for sustainable learning in CPPB? What kinds of processes have been build-in to ensure follow ups?

4. Revision: Main Points Structure of Guidance New Overall Structure
needs to serve purpose of audiences
Evaluation Managers
Evaluators (DEV + CPPB)
Broader CPPB field
New Overall Structure
Introduction of CPPB Context
Introduction into evaluations
General
Specificities of CPPB evaluation (incl. conflict sensitivity as eval goal, transversal theme or conduct issue
Managing/Preparing an Evaluation
Conducting an Evaluation
Preconditions for Evaluations -> Planning for Results, Evaluability and closing strategic gap

5. Revision: Main Points
Chapter 2 Managing/Preparing an Evaluation: Points to be added/changed
Evaluation’s general focus
Evaluation Criteria (DAC + 3C) + transversal themes (e.g. conflict sensitivity, gender)
Process Design
Elaborate on Conflict Analysis Topic
Build-in Quality Control
Phases + Reporting (distinction along Types of eval)
Politics and other real world risks
Build-in Reference/Steering group + ombudsperson
Feedback, Dissemination, Learning etc
TORs + how they will be adapted after inception phase!
(Flexible) budgets
Request (potential) evaluators to develop proposal for evaluation design, approaches, methodologies, feasibility

6. Revision: Main Points
Chapter 3: Conducting an Evaluation: much more focus on HOW
Overall evaluation designs
Distinction between different types + scopes of evaluations
Evaluation Approaches: purposes + HOW+ best practice
Linking elements + methodology to criteria
Relevance: need for conflict analysis + theory of change + HOW to do it with a set of options + examples (incl. sampling)
Effectiveness (theory of change), etc.
Clarification about impact assessment (impact versus outcomes versus conflict effects)
Core challenges
Data gathering under constraints including overreliance on interviews + reality based options on HOW incl. sequences + feasibility
Politics

7. Revision: Main Points Conflict / Context Analysis Transparency about
HOW, Ownership and USE in Evaluation
Adjusting Types of Analysis to Evaluation Goals
Elements
Historical, Socio-economic context, etc
National + local level
Conflict Analysis insufficient, more elements needed
Analysis of Peacebuilding Context and short, medium and long-term needs
Assessing conflict sensitivity of activities: general + context specific definition and assessment (+ options for how, ex. Coverage/partners, power relations => link to conflict analysis)
Assessing conflict monitoring capacity/performance
Assessing adaptation capacities
‘Conflict’ is not always the good term!!!

8. Sustainability Draft Guidance follow ups
Revision
Dissemination in different forms
Capacity Building/Training for Eval Managers + (potential) Evaluators
Work on Evaluation Culture
Awareness Building in different communities
DAC EVAL Net to draft harmonized SUPER Guidance
How to use the Guidance for Learning in CPPB
INCAF to make use of policy lessons
Ongoing feedback loop needs to be institutionalised