1. Outcome Mapping Master-Class Outcome Harvesting: a tool for identifying and understanding results in complex circumstances Ricardo Wilson-Grau Brussels, Belgium 25 March 2013

2. Introductions Please mention your name and organisation and briefly describe in a minute or two what is your experience in outcome monitoring and evaluation. ricardo.wilson-grau@inter.nl.net

3. Purpose and intended results 1.You will learn how to apply the essential principles of the six steps of Outcome Harvesting. 2.At the end of the day, you will be able to design a monitoring process and formative or summative evaluations using the tool. ricardo.wilson-grau@inter.nl.net

4. Methodology Personal reflection Conceptual presentation Exercise with case Questions & Answers ricardo.wilson-grau@inter.nl.net

5. Agenda ✓ 9:15Introductions 9:30The complex challenges of monitoring and evaluating outcomes 10:45Coffee 11:00Primary users and principal uses 12:00Useable Outcome Harvesting questions 13:00Lunch 14:00Identifying and formulating outcomes 15:30Coffee 15:45Substantiation, analysis, interpretation and use 16:30Summing up 16:45Evaluation ricardo.wilson-grau@inter.nl.net

6. The complex challenges of monitoring and evaluating outcomes

7. Inspired by the Outcome Mapping methodology. A tool for practitioners operating in situations of considerable uncertainty to monitor and evaluate the social change results they are achieving. Outcome Harvesting ricardo.wilson-grau@inter.nl.net

8. Outcome Harvesting Developed since 2003 by me and my colleagues Barbara Klugman, Claudia Fontes, Fe Briones Garcia, Gabriela Sánchez, Goele Scheers, Heather Britt, Jennifer Vincent, Julie Lafreniere, Juliette Majot, Marcie Mersky, Martha Nuñez, Mary Jane Real, Natalia Ortiz and Wolfgang Richert. ricardo.wilson-grau@inter.nl.net

9. 1. International social change networks ricardo.wilson-grau@inter.nl.net

10. International development funders ricardo.wilson-grau@inter.nl.net

11. Over 300 networks and associations, NGOs, community- based organisations, research institutes, and government agencies on all seven continents. Identifying and reporting 2,500+ changes they have influenced. ricardo.wilson-grau@inter.nl.net

12. What did all of these organisations have in common?

13. PlanningMonitoring “I can honestly say that not a day goes by when we don’t use those evaluations in one way or another.” Evaluation ricardo.wilson-grau@inter.nl.net

14. PlanningMonitoring “I can honestly say that not a day goes by when we don’t use those evaluations in one way or another.” Evaluation ricardo.wilson-grau@inter.nl.net

15. Why?

16. Personal reflection In one minute each, explain to the person next to you what you understand as “complexity”. Please hold your reflections and allow me to explain how complexity is understood in Outcome Harvesting. Then, we will close the circle with the Questions & Answers. ricardo.wilson-grau@inter.nl.net

17. Relationships of cause and effect are KNOWN Relationships of cause and effect are UNKNOWN Simple Complex ricardo.wilson-grau@inter.nl.net

18. ACTIVITIES OUTCOMES IMPACT INPUTS Inspired by Jeff Conklin, cognexus.org Time OUTPUTS Simple intervention A polio vaccination campaign ricardo.wilson-grau@inter.nl.net

19. ACTIVITIES OUTCOMES IMPACT INPUTS Inspired by Jeff Conklin, cognexus.org Time Simple intervention OUTPUTS A polio vaccination campaign ¿Are we effective? Are we efficient? ricardo.wilson-grau@inter.nl.net

20. Ten Steps to Designing, Building and Sustaining a Results- Based Monitoring and Evaluation System Agreeing on outcomes to monitor and evaluate Baseline data on indicators Monitoring for results Reporting your findings Sustaining the M&E system within your organization 1 23456 7 9 10 8 Conducting a readiness assessment Selecting key indicators to monitor outcomes Planning for improvement — selecting results targets Using evaluation information Using your findings When you can plan with relative certainty, conventional evaluation works ricardo.wilson-grau@inter.nl.net

21. Implementing a new approach to address domestic violence OUTPUT OUTCOME INPUTS ACTIVITY INPUTS ACTIVITY INPUTS ACTIVITY INPUTS OUTPUT ACTIVITY OUTPUT OUTCOME But in complex interventions Time ricardo.wilson-grau@inter.nl.net

22. 22 OUTPUT OUTCOME INPUTS ACTIVITY INPUTS ACTIVITY INPUTS ACTIVITY INPUTS OUTPUT ACTIVITY OUTPUT OUTCOME Time Dimensions in which the relationships of cause and effect are known SIMPLE Not blackand white ricardo.wilson-grau@inter.nl.net

23. Not black and white 23 OUTPUT OUTCOME INPUTS ACTIVITY INPUTS ACTIVITY INPUTS ACTIVITY INPUTS OUTPUT ACTIVITY OUTPUT OUTCOME Time COMPLEX Unknown relations of cause and effect dominate. Results are substantially unforeseeable. ricardo.wilson-grau@inter.nl.net

24. Thanks to Michael Quinn Patton ricardo.wilson-grau@inter.nl.net

25. Exercise #1 ➤ Now we will begin to use the simulated case. The idea is for you to imagine that FundNow described in this page and a half profile is an NGO in your country. You have the power to decide what to do in order to achieve its multi-annual objectives. ➤ First, however, you want to assess the degree of complexity you face. So please classify on this sheet each objective on a scale from 1 for very certain (or simple) to 7 for highly uncertain (or complex) ➤ It is a matter of judgement. There are really no right or wrong answers. ➤ And only rank the objectives for which you feel comfortable. ricardo.wilson-grau@inter.nl.net

27. 27 Evaluation and complexity Factors of Complexity COMPLEXIMETER High Low Disagreement About what is the development challenge or its solution Uncertainty About what will be the results of your actions to solve the development challenge ricardo.wilson-grau@inter.nl.net

28. TIME Planning 28 MonitoringEvaluation COMPLEXIMETER PME and complexity in time ricardo.wilson-grau@inter.nl.net

29. TIME MonitoringEvaluation Planning COMPLEXIMETER PME and complexity in time ricardo.wilson-grau@inter.nl.net

30. TIME EvaluationMonitoring Planning COMPLEXIMETER PME and complexity in time ricardo.wilson-grau@inter.nl.net

31. TIME 31 COMPLEXIMETER MonitoringEvaluation Planning PME and complexity in time ricardo.wilson-grau@inter.nl.net

32. In sum The greater the complexity the less important is planning and the more vital it is to monitor and evaluate in real time, in order to know what you are achieving, what is working and what you should do differently. ricardo.wilson-grau@inter.nl.net

33. Outcome Harvesting is NOT useful ACTIVITIES OUTCOMES IMPACT INPUTS Time OUTPUTS ricardo.wilson-grau@inter.nl.net

34. Outcome Harvesting IS useful OUTPUT OUTCOME INPUTS ACTIVITY INPUTS ACTIVITY INPUTS ACTIVITY INPUTS OUTPUT ACTIVITY OUTPUT OUTCOME Time ricardo.wilson-grau@inter.nl.net

35. Questions & Answers How compatible or incompatible is your understanding of complexity now with what I have explained of complexity for Outcome Harvesting? Any other questions about the complex challenges of monitoring and evaluating outcomes? ricardo.wilson-grau@inter.nl.net

36. Agenda ✓ ✓ ✓ 9:15Introductions 9:30The complex challenges of monitoring and evaluating outcomes 10:45Coffee 11:00Primary users and principal uses 12:00Useable Outcome Harvesting questions 13:00Lunch 14:00Identifying and formulating outcomes 15:30Coffee 15:45Substantiation, analysis, interpretation and use 16:30Summing up 16:45Evaluation ricardo.wilson-grau@inter.nl.net

37. Outcome Harvesting Six steps: 1.Design the harvest 2.Review documentation and draft outcomes 3.Engage with informants 4.Substantiate 5.Analyse, interpret 6.Support use of findings ricardo.wilson-grau@inter.nl.net

38. SOCIAL ACTOR Individual Organisation Institution Group or community The essence I A. Outcome defined as a change in a social actor ricardo.wilson-grau@inter.nl.net

39. SOCIAL ACTOR Individual Organisation Institution Behaviour Relationships Actions, activities Policies and practices Group or community CHANGING The essence I A. Outcome defined as a change in a social actor ricardo.wilson-grau@inter.nl.net

40. CauseB. Contribution: Effect The essence II Detective Arqueologist Epidemiologist Auto mechanic ricardo.wilson-grau@inter.nl.net

41. From focusing on what one does For M&E, Outcome Harvesting represents a paradigm change ricardo.wilson-grau@inter.nl.net

42. To focusing on what one achieves For M&E, Outcome Harvesting represents a paradigm change ricardo.wilson-grau@inter.nl.net

43. Outcome Harvesting Six steps: 1.Design the harvest 2.Review documentation and draft outcomes 3.Engage with informants 4.Substantiate 5.Analyse, interpret 6.Support use of findings ricardo.wilson-grau@inter.nl.net

44. Personal reflection Always assuming that FundNow is an organisation in your country, write down who you consider to be its stakeholders. We will come back to your list in the exercise. ricardo.wilson-grau@inter.nl.net

45. 1. Design the harvest to meet needs It is tempting to collect data on everything a project, programme or organisations does and achieves, but it is not practical. “At IDRC, if you cannot identify and articulate the primary intended users and uses of the evaluation you should not conduct the evaluation. Unused evaluation is a waste of precious human and financial resources.” ricardo.wilson-grau@inter.nl.net

46. Primary intended users Principal intended uses ricardo.wilson-grau@inter.nl.net

47. Primary intended users Those people who need the harvest or its findings in order to make decisions or take actions. The users are actively involved. The harvester and her commissioner agree together who are the primary users, and then what are their principal uses. ricardo.wilson-grau@inter.nl.net

48. Principal uses There can be use for the process itself of harvesting As well as for the findings of the harvest. ricardo.wilson-grau@inter.nl.net

49. Typical process uses Improve communication and foment common understanding Strengthen the implementation of a project or programme because “what is measured tends to be what is gets done” Increase participation and ownership Foster a learning culture ricardo.wilson-grau@inter.nl.net

50. Common uses of findings Inform what was done and is being achieved Take decisions to modify the model or initiative Plan future work Evaluate the performance ricardo.wilson-grau@inter.nl.net

51. Users vs. Audience Those people who need the harvest or its findings in order to make decisions or take actions. The users are actively involved. Users are different from the broader audience of stakeholders who may see and react to the harvest’s results. The audience is passively interested in the harvest. ricardo.wilson-grau@inter.nl.net

52. Exercise #2 With the same person with whom you had just had the personal reflection, exemplify whom you both consider would be FundNow’s users of an Outcome Harvest and what might be their uses? Who would be the audience? ricardo.wilson-grau@inter.nl.net

53. FundNow Users vs. Audience Primary users: They will be involved in decision-making through all six steps of the harvest Audience: They may be informed periodically of progress but not necessarily. They may be consulted on how to use the findings but they are not responsible for the use. ricardo.wilson-grau@inter.nl.net

54. Questions & Answers Do you have any questions about primary users and principal uses of an Outcome Harvest and why it is important to distinguish them from the broader audience for the harvest findings and other stakeholders? ricardo.wilson-grau@inter.nl.net

55. Agenda ✓ ✓ ✓ ✓ 9:15Introductions 9:30The complex challenges of monitoring and evaluating outcomes 10:45Coffee 11:00Primary users and principal uses 12:00Useable Outcome Harvesting questions 13:00Lunch 14:00Identifying and formulating outcomes 15:30Coffee 15:45Substantiation, analysis, interpretation and use 16:30Summing up 16:45Evaluation ricardo.wilson-grau@inter.nl.net

56. Personal reflection If you were to evaluate FundNow, in a sentence write one evaluation question that you would consider important for the use(s) of the primary user(s). You will now use your question either to further craft it, or completely re-write it, as we see the purpose of useable questions in Outcome Harvesting. ricardo.wilson-grau@inter.nl.net

57. Primary intended users Principal intended uses Useable Outcome Harvesting questions ricardo.wilson-grau@inter.nl.net

58. Evaluation findings Adapted from: A Practical Guide for Engaging Stakeholders in Developing Evaluation Questions, Robert Wood Johnson Foundation, 2009 Useable harvest questions RelevantCrediblePriority ricardo.wilson-grau@inter.nl.net

59. Useable harvest questions What questions must be answered so that the primary users will be able to make decisions or take other action? For a harvest question to be useable, it must provide both : Information: What data has to be collected? Plus Interpretation: So what does the data mean? ricardo.wilson-grau@inter.nl.net

60. Example of a What question What were FundNow’s outcomes in 2011-12? We have 12 outcomes in two years! Fantastic. What does it mean? ricardo.wilson-grau@inter.nl.net

61. Example of a So What question What were FundNow’s outcomes in 2011-12? What is the relative importance of FundNow’s outcomes in 2011-12 for increasing the quantity and quality of the work of women’s organisations in the country? A useable question is an evaluative question ricardo.wilson-grau@inter.nl.net

62. Exercise #3 In groups of twos, agree a different, useable outcome harvest question for FundNow, always in one sentence. ricardo.wilson-grau@inter.nl.net

63. Actionable (useable) evaluation questions 1. Was the programme—and is it still—needed? How well does it address the most important root causes? Is it still the right solution? 2. How well designed and implemented is the programme? 3. How valuable are the outcomes for [insert impactee group]? 4. What works best for whom, under what conditions, and why/how? 5. How worthwhile was it overall? Or, which parts or aspects of the programme generated the most valuable outcomes for the time, money and effort invested? 6. How sustainable is the impact? How sustainable is the programme itself? Source: Jane Davidson, Actionable Evaluation Basics, RealEvaluation.com, 2012 ricardo.wilson-grau@inter.nl.net

64. Questions & Answers Do you have any questions about formulating useable “What?” PLUS “So What?” Outcome Harvesting questions? ricardo.wilson-grau@inter.nl.net

65. Agenda ✓ ✓ ✓ ✓ ✓ 9:15Introductions 9:30The complex challenges of monitoring and evaluating outcomes 10:45Coffee 11:00Primary users and principal uses 12:00Useable Outcome Harvesting questions 13:00Lunch 14:00Identifying and formulating outcomes 15:30Coffee 15:45Substantiation, analysis, interpretation and use 16:30Summing up 16:45Evaluation ✓ ricardo.wilson-grau@inter.nl.net

66. Outcome Harvesting Six steps: 1.Design the harvest 2.Review documentation and draft outcomes 3.Engage with informants 4.Substantiate 5.Analyse, interpret 6.Support use of findings ricardo.wilson-grau@inter.nl.net

67. Personal reflection For all of you, what makes monitoring and evaluation data credible? I will make a list here on the flip chart. ricardo.wilson-grau@inter.nl.net

68. Primary intended users Principal intended uses Useful Outcome Harvesting questions Credible data ricardo.wilson-grau@inter.nl.net

69. Select the most credible sources People who personally know who changed and what changed. They give initial authenticity, confidence and credibility to the information you harvest. Thus, Outcome Harvesting is highly participatory. ricardo.wilson-grau@inter.nl.net

70. Historical documentation Person(s) who influenced others in time Independent third parties SOURCES Triangulation of sources ricardo.wilson-grau@inter.nl.net

71. Exercise #4 Please take five minutes to read this 2012 annual report from FundNow. Then, with the person seated next to you: A. Identify one outcome: Who changed behaviour, relationships, activities, actions, policies or practice? What changed? When did it change? And where? ricardo.wilson-grau@inter.nl.net

72. FundNow 2012 Annual Report Feedback from our allies and co-workers about FundNow’s activities in 2012 has been very positive, emphasizing the usefulness of our research publications to inform about the funding challenges and opportunities facing women's organizations in the country. Our research has shown that women's rights organizations have been living in a form of "resist and survive" mode but doing amazing work for the advancement of women's rights. People always mention our online newsletter FundNow as a tool that has helped them to think more strategically about raising funds for their organizations. The preparatory research we did for each one of the13 meetings we convened was greatly appreciated. This year, our innovative research on trends in financing for gender equality and women's organization has been a strongly supportive tool for women’s groups obtaining more and better resources across the country. Based on the collection of "gems" of impact of projects implemented by the grant recipients of central government’s National Fund for MDG3 (referring to the third Millennium Development Goal which is on gender equality), we compiled a report that has been used by gender activists. For example, donors internally to promote resource allocation for gender equality and women's rights. For example, an ad hoc group in the ministry responsible for the National Fund for MDG3 used our compilation in their lobbying internally for the replenishment of the Fund, which resulted in the addition of US$4 million to the Fund in early 2012 and a commitment to launch a new phase of the Fund with US$20 million in 2013. In addition, this year we have successfully organized in two provinces six strategic meetings for women's organizations on provincial resource mobilization. (In addition, we organised t nationally for a total of 13.) These were the first such meetings in those provinces. In one province 5 funders and 18 women's organizations attended and in the other 4 and 12, respectively. The background research we did based on regional surveys in preparation for the meetings was very well received and participants said we were filling a clear gap of information with our findings. Many participants expressed that these meetings are a strategic opportunity to establish new contacts, engage in networking, develop strategies, offer a space for potential donors and grantees to meet and share experiences. Everyone said the meetings enabled them to better understand the context in which both funders and women's organizations operate. The meeting participants also had time to reflect on their work and said these events served as inspiration. All the participants expressed their commitment to work together in the future to guarantee funding for women's organizations in the two provinces. ricardo.wilson-grau@inter.nl.net

73. MGD3 Fund outcome A. Description: During 2011, the ministry decided to allocate US$4 million to the National Fund for MDG3 in early 2012 and committed to launch a new phase of the Fund with US$20 million in 2013, to strengthen the rights and opportunities for women and girls. WHO? ricardo.wilson-grau@inter.nl.net

74. WHAT? MGD3 Fund outcome A. Description: During 2011, the ministry decided to allocate US$4 million to the National Fund for MDG3 in early 2012 and committed to launch a new phase of the Fund with US$20 million in 2013, to strengthen the rights and opportunities for women and girls. ricardo.wilson-grau@inter.nl.net

75. WHAT? MGD3 Fund outcome A. Description: During 2011, the ministry decided to allocate US$4 million to the National Fund for MDG3 in early 2012 and committed to launch a new phase of the Fund with US$20 million in 2013, to strengthen the rights and opportunities for women and girls. ricardo.wilson-grau@inter.nl.net

76. WHEN? MGD3 Fund outcome A. Description: During 2011, the ministry decided to allocate US$4 million to the National Fund for MDG3 in early 2012 and committed to launch a new phase of the Fund with US$20 million in 2013, to strengthen the rights and opportunities for women and girls. ricardo.wilson-grau@inter.nl.net

77. WHERE? MGD3 Fund outcome A. Description: During 2011, the ministry decided to allocate US$4 million to the National Fund for MDG3 in early 2012 and committed to launch a new phase of the Fund with US$20 million in 2013, to strengthen the rights and opportunities for women and girls. ricardo.wilson-grau@inter.nl.net

78. Exercise #5 B. How did FundNow contribute to the ministry deciding to allocate US$4 million to the National Fund for MDG3 in early 2012 and committed to launch a new phase of the Fund with US$20 million in 2013? What activities and outputs plausibly contributed to the change in the social actor, however partially, indirectly and even unintentionally? ricardo.wilson-grau@inter.nl.net

79. Plausible? MGD3 Fund outcome A. Description: During 2011, the ministry decided to allocate US$4 million to the National Fund for MDG3 in early 2012 and committed to launch a new phase of the Fund with US$20 million in 2013, to strengthen the rights and opportunities for women and girls. B. Contribution: FundNow compiled research that was used by an ad hoc group within the ministry for internal lobbying for the replenishment of the MDG3 Fund. ricardo.wilson-grau@inter.nl.net

80. Other information? Significance of the outcome Collaboration with other social actors Contribution of other actors and factors History Context Evidence of impact on people’s lives And so forth. ricardo.wilson-grau@inter.nl.net

81. 81 Less is more! When? Who? What? Where? Harvesting outcomes ricardo.wilson-grau@inter.nl.net

82. Outcome Harvesting Six steps: 1.Design the harvest 2.Review documentation and draft outcomes 3.Engage with informants 4.Substantiate 5.Analyse, interpret 6.Support use of findings ricardo.wilson-grau@inter.nl.net

83. Historical documentation Person(s) who influenced others in time Independent third parties SOURCES Triangulation of sources ricardo.wilson-grau@inter.nl.net

84. METHODS Triangulation of methods ricardo.wilson-grau@inter.nl.net

85. Comment [RW-G3]: Is this an appropriate characterisation of the purpose? Can you make it more measurable? Comment [RW-G2]: Can you be more concrete? Which ministry? Who in the ministry? Comment [RW-G1]: Can you be more specific about the date — when in 2011, which month and day if possible? Description: During 2011, the ministry decided to allocate US$4 million to the National Fund for MDG3 in early 2012 and committed to launch a new phase of the Fund with US$20 million in 2013, to strengthen the rights and opportunities for women and girls. ricardo.wilson-grau@inter.nl.net

86. In sum The description of the outcome and the contribution have to be sufficiently specific and concrete so that they are: Plausible Verifiable Detailed enough to meet the primary users’ principal uses. ricardo.wilson-grau@inter.nl.net

87. Questions & Answers Do you have any questions about the content of a description of an outcome and the contribution to it? ricardo.wilson-grau@inter.nl.net

88. Agenda ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 9:15Introductions 9:30The complex challenges of monitoring and evaluating outcomes 10:45Coffee 11:00Primary users and principal uses 12:00Useable Outcome Harvesting questions 13:00Lunch 14:00Identifying and formulating outcomes 15:30Coffee 15:45Substantiation, analysis, interpretation and use 16:30Summing up 16:45Evaluation ricardo.wilson-grau@inter.nl.net

89. Outcome Harvesting Six steps: 1.Design the harvest 2.Review documentation and draft outcomes 3.Engage with informants 4.Substantiate 5.Analyse, interpret 6.Support use of findings ricardo.wilson-grau@inter.nl.net

90. Which outcomes and how many to substantiate depends on what is required for the credibility of the findings. Outcome: To what degree are you in agreement with the description of the ministry’s decision to expand its funding of MDG 3? [ ] Fully agree [ ] Partially agree [ ] Disagree Comments if you like: 4. Substantiate: Verify accuracy + deepen understanding ricardo.wilson-grau@inter.nl.net

91. 5. Analyse and interpret the outcomes to answer the useable Outcome Harvesting questions ricardo.wilson-grau@inter.nl.net

92. Methodologist 1 or 2 content experts Person(s) who influenced others in time ANALYSTS Triangulation of analysis ricardo.wilson-grau@inter.nl.net

93. Excel Database ricardo.wilson-grau@inter.nl.net

94. Access Database ricardo.wilson-grau@inter.nl.net

95. Seeking processes, patterns, trends in what happened ricardo.wilson-grau@inter.nl.net

96. Women’s Land Inheritance Rights in an African Country President declares women's land inheritance a right Ten widows take late husbands’ family to court Court decides landmark case in favour of widow Agricultural union admits widows as members Religious leader declares women's land inheritance a right Policy change Legislature passes women’s right to land inheritance law Agriculture committee discusses draft law Women’s group launches campaign Legislator drafts law Widow takes late husband’s family to court For the first time, press runs story on women’s land inheritance Finance committee discusses draft law Court decides against widow Late husbands’ families murder three widows Women’s group initiates legal aid for widows Over the past 5 years, in 90% of the cases the Land Commission has given to widows the titles of their late husbands’ land Practice change Land Commission approves regulations for women’s inheritance of deceased husband’s land YEARS 1-3YEARS 4-6YEARS 7-10 ricardo.wilson-grau@inter.nl.net

97. So what does it all add up to? ricardo.wilson-grau@inter.nl.net

98. IMPACT Theory of change RESULTS SOCIAL ACTORS ASSUMPTIONS MISSION STRATEGIES ricardo.wilson-grau@inter.nl.net

99. Systems approach Peter Woodrow, CDA ricardo.wilson-grau@inter.nl.net

100. 6. Support for use of findings ricardo.wilson-grau@inter.nl.net

101. Questions & Answers Do you have any questions about the three last steps? 4. Substantiate 5. Analyse, interpret 6. Support use of findings ricardo.wilson-grau@inter.nl.net

102. Agenda ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 9:15Introductions 9:30The complex challenges of monitoring and evaluating outcomes 10:45Coffee 11:00Primary users and principal uses 12:00Useable Outcome Harvesting questions 13:00Lunch 14:00Identifying and formulating outcomes 15:30Coffee 15:45Substantiation, analysis, interpretation and use 16:30Summing up 16:45Evaluation ricardo.wilson-grau@inter.nl.net

103. Methodological challenges Participation Focusing on what one achieved Expressing oneself in writing Coaching Everyone keeping to the deadlines Working as an archaeologist or forensic scientist or police detective Recognising what did not change as an outcome Not all outcomes are positive Harvesting best done by two or more people Supporting use of findings ricardo.wilson-grau@inter.nl.net

104. Final Questions and Answers? ricardo.wilson-grau@inter.nl.net

105. Agenda ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 9:15Introductions 9:30The complex challenges of monitoring and evaluating outcomes 10:45Coffee 11:00Primary users and principal uses 12:00Useable Outcome Harvesting questions 13:00Lunch 14:00Identifying and formulating outcomes 15:30Coffee 15:45Substantiation, analysis, interpretation and use 16:30Summing up 16:45Evaluation ricardo.wilson-grau@inter.nl.net

106. Evaluation of workshop I would like to ask you to take a few minutes to reflect on the extent to which we accomplished the intended results. Please make suggestions on what I can do differently to improve this workshop. Also, is there anything else that you believe I may be able to do to support you? Ricardo.Wilson-Grau@inter.nl.net ricardo.wilson-grau@inter.nl.net

107. Many thanks! Ricardo.Wilson-Grau@inter.nl.net