1. outcomemapping.ca Complexity and Outcome Harvesting

2. Most recently In June the World Bank published 10 case studies and a toolkit for using Outcome Harvesting 2 http://www.outcomemapping.ca/res ource/resource.php?id=452 http://wbi.worldbank.org/wbi/document /cases-outcome-harvesting

3. e Harvesting

4. outcomemapping.ca e Harvesting 4 Inspired and informed by the Outcome Mapping methodology. A tool for practitioners operating in dynamic, uncertain situations to monitor and evaluate the social change results they are achieving.

5. Outcome Harvesting Six steps: 1.Design the harvest — users => uses => evaluation questions => data to be collected 2.Review documentation and draft outcomes => intervention’s contribution 3.Engage with informants — generally within the intervention 4.Substantiate with independent but knowledgeable third parties 5.Analyse, interpret — What? + So What? but not Now What? 6.Support use of findings TRIANGULATION

6. outcomemapping.ca International social change networks 6

7. outcomemapping.ca International development funders 7

8. outcomemapping.ca What did all of these organisations have in common?

9. outcomemapping.ca Relationships of cause and effect are KNOWN Simple 9

10. Annual polio vaccination campaign ACTIVITIES OUTCOMES IMPACT INPUTS Inspired by Jeff Conklin, cognexus.org Time OUTPUTS M&E of a simple intervention 10

11. outcomemapping.ca Relationships of cause and effect are UNKNOWN Complex 11

12. OUTPUT OUTCOME INPUTS ACTIVITY INPUTS ACTIVITY INPUTS ACTIVITY INPUTS OUTPUT ACTIVITY OUTPUT OUTCOME M&E of a complex intervention Time Strengthening nonviolent responses to communal conflict in the Horn of Africa 12

13. outcomemapping.ca OUTPUT OUTCOME INPUTS ACTIVITY INPUTS ACTIVITY INPUTS ACTIVITY INPUTS OUTPUT ACTIVITY OUTPUT OUTCOME Time Dimensions in which the relationships of cause and effect are known NOT EITHER OR SIMPLE 13

14. outcomemapping.ca OUTPUT OUTCOME INPUTS ACTIVITY INPUTS ACTIVITY INPUTS ACTIVITY INPUTS OUTPUT ACTIVITY OUTPUT OUTCOME Time COMPLEX Results are substantially unforeseeable. Unknown relations of cause and effect dominate. NOT EITHER OR 14

15. outcomemapping.ca Outcome Harvesting and complexity The greater the: 1. Disagreement about what is the development challenge 2. Disagreement about what is its solution 3. Uncertainty about what will be the results of your actions to solve the development challenge The more Outcome Harvesting may be useful COMPLEXIMETER High Low 15

16. 16 Formative evaluation Summative evaluation Developmental evaluation Developmental Evaluation is not, however, the same as evaluation of development.

17. outcomemapping.ca When do you evaluate? Time SummativeDevelopmentalFormative Progress of intervention 17

18. outcomemapping.ca Objectives or Outcomes Outputs Long-term Goal (Impact) ActivitiesInputs RESULTS BASED M&E STEP 1: Vision INTENTIONAL DESIGN STEP 2: Mission STEP 3: Boundary Partners STEP 4: Outcome Challenges STEP 5: Progress Markers STEP 6: Strategy Maps Indicators

19. outcomemapping.ca Objectives or Outcomes Outputs Long-term Goal (Impact) ActivitiesInputs STEP 1: Vision STEP 2: Mission STEP 3: Boundary Partners STEP 4: Outcome Challenges STEP 5: Progress Markers STEP 6: Strategy Maps Indicators GLOBAL PARTNERSHIP FOR THE PREVENTION OF ARMED CONFLICT

20. Objectives or Outcomes Outputs Long-term Goal (Impact) ActivitiesInputs STEP 1: Vision STEP 2: Mission STEP 3: Boundary Partners STEP 4: Outcome Challenges STEP 5: Progress Markers STEP 6: Strategy Maps Indicators ICT4D in Asia PAN and Africa ACACIA Nigeria Evidence‐based Health System Initiative EcoHealth Fieldbuilding Leadership Initiative Consorcio por la Salud, Ambiente y Desarrollo Regional Peacebuilding Programme in the Horn of Africa

21. outcomemapping.ca Objectives or Outcomes Outputs Long-term Goal (Impact) ActivitiesInputs STEP 1: Vision STEP 2: Mission STEP 3: Boundary Partners STEP 4: Outcome Challenges STEP 5: Progress Markers STEP 6: Strategy Maps Indicators Art and Culture Programme in Central America

22. outcomemapping.ca In sum Outcome Harvesting is “an evaluation approach that does not measure progress towards predetermined outcomes, but rather collects evidence of what has been achieved, and works backward to determine whether and how the project or intervention contributed to the change.” - UNDP 22 https:// undp.unteamworks.org/node/370238

23. outcomemapping.ca Discussion Have any of you found other ways to apply the principles of Outcome Mapping to evaluate projects, programmes or organisations when the elements of intentional design have been missing?