1. outcomemapping.ca

2. Introduction to Outcome Mapping Simon Hearn, Julius Nyangaga, Ziad Moussa Brussels, Belgium March 25-26, 2013

4. Workshop Objectives Clarify what Outcome Mapping is and does Introduce and apply Outcome Mapping concepts and tools Consider if and how Outcome Mapping could be useful in our work

5. Agenda Overview This morning : Key concepts Vision Mission Boundary Partners This afternoon: Outcome Challenge Progress Markers Strategy Map Tomorrow : Organisational Practices M&E planning and journals

6. Proposed process guidelines: Everyone can participate One speaks, all listen Value each other’s ideas No smoking or mobile devises Respect each other by being on time Questions any time (answers may come later) Enjoy... Network... Laugh... Sing..!...Other?

7. If your M&E processes were practical and useful, what would they do, what characteristics would they have?

8. Key concepts in Outcome Mapping

9. Acknowledgements This presentation makes use of various materials that were shared by members of the global OM community. Without being exhaustive, special thanks goes to Terry Smutylo, Steff Deprez, Jan Van Ongevalle, Robert Chipimbi, Daniel Roduner, Kaia Ambrose, Ricardo Wilson-Grau and many others.

10. Source: A guide for project M&E: IFAD

11. Time Source: Ricardo Wilson-Grau (inspired by Jeff Conklin) Conventional thinking… ACTIVITIES OUTPUTS OUTCOMES IMPACT INPUTS

12. Time … clashes with relationships of cause and effect that are unknown Source: Ricardo Wilson-Grau (inspired by Jeff Conklin) ACTIVITIES OUTPUTS OUTCOMES IMPACT INPUTS

13. Social change can be… Complex: involve a confluence of actors and factors Unstable: independent of project duration Non-linear: unexpected, emergent, discontinuous Two-way: intervention may change Beyond control: but subject to influence Incremental, cumulative: watersheds & tipping points Source: Terry Smutylo

14. OUTPUT OUTCOME ACTIVITY INPUTS ACTIVITY INPUTS ACTIVITY INPUTS OUTPUT ACTIVITY OUTPUT OUTCOME Source: Ricardo Wilson-Grau Time INPUTS

15. Brief definition of OM A participatory method for planning, monitoring and evaluation Focused on changes in behaviour of those with whom the project or program works Oriented towards social & organizational learning

16. OM’s answer Source: Terry Smutylo / OM Lab 2012 Support people to build their own well-being Enable interventions to adapt as they engage Apply a systems understanding Start from observable behaviour change Recognise that all interventions have limited influence Embrace different perspectives

17. A brief history of OM 1990s: post-Rio need to demonstrate ‘sustainable’ results 1998: Barry Kibel and Outcome Engineering 1999: Methodological collaboration with projects 2000: Publication of manual in English 2002: Training, facilitation & usage globally 2006: OM Learning Community 2008: CLAMA 2010: East Africa and beyond

19. “Being attentive along the journey is as important as, and critical to, ariving at the destination ” Michael Quinn Patton

20. Three key concepts in OM: 1.Sphere of influence 2.Boundary Partners 3.Outcomes understood as changes in behaviour

21. There is a limit to our influence Project People the project works with/through People who benefit from the project Sphere of control Sphere of influence Sphere of concern

22. There is a limit to our influence Inputs, activities, outputs Outcomes: Changes in behavior Impact: Changes in state Sphere of control Sphere of influence Sphere of concern

23. Participatory research on demonstration farms to develop approaches to drip irrigation Farmers participate in field trials Participating farmers learn how to use drip irrigation equipment Extension workers visit demonstration farms Training of extension workers Publication of performance of different set- ups Increased knowledge of techniques Extension workers promoting drip irrigation Farmers adopting drip irrigation methods Reduced numbers of new wells Greater quantities of groundwater available Source: Terry Smutylo

24. Who are your boundary partners? Programme BeneficiariesStakeholdersBoundary Partners

26. Focus of Outcome Mapping Outcome Mapping Community ownership increases Program influence decreases InputsActivitiesOutputsOutcomesImpacts

28. ✓ OM is a guide to the journey we take with our partners. We co-create the map. ✓ It focuses on the intention and what happens along the way ✓ The map is not the territory, it shows the route taken ✓ “The only real voyage of discovery exists, not in seeing new landscapes, but in having new eyes” (Marcel Proust) Where is the map?

29. Step 1: Vision

30. improved human, social, & environmental wellbeing

31. I have a dream! Martin Luther King, Jr. August 28, 1963

32. A vision statement.. Guides Motivates and inspires Is an ‘accountability-free zone’

33. 33 Vision in graphic form, Nagaland (India)

34. Vision statement in narrative form Local authorities, communities, and international organizations in developing countries in Africa recognize the value of HIV/AIDS intervention as an integral part of social & economic development. Municipal, regional, and national governments actively support HIV/AIDS prevention activities by formulating and implementing effective public health policies. Using research findings, they have developed a comprehensive public health strategy to slow down the infection rate. Formerly marginalized groups (e.g. women and youth) are organized into advocacy groups that can effectively formulate their needs to policy makers. All groups have access to reliable and relevant technical information about HIV/AIDS prevention and are able to make informed choices. In essence, there are healthier, happier, and wealthier communities.

35. Vision facilitation question Imagine that, 5-10 years from now, the program has been extremely successful. Things have improved beyond your most ambitious dreams. What changes have occurred? What (& how) are your intended beneficiaries doing? What are your partners doing? Describe the better world you are seeking.

36. Step 2: Mission

37. The mission is that “bite” of the vision statement on which the program is going to focus.

38. How the program intends to apply its resources in support of the vision The areas in which the program intends to work How the program will support the achievement of outcomes by its direct partners A mission statement describes:

39. Example Mission Statement In support of this vision and on behalf of its donors, the program will work in the areas of research, dissemination, capacity building, & coordination. It will contribute to the production, synthesis, & dissemination of research data, position papers, & other information that will sensitize local & international actors to HIV/AIDS prevention. The program will seek to expand the range of disciplines involved in HIV/AIDS research. It will enhance HIV/AIDS research capacity in order to produce credible information for local, national, & international policy-making & program development. It will promote an interest in HIV/AIDS research among new researchers by providing research fellowships, mentorship, & training opportunities. The program will contribute to the development of linkages between Northern & Southern researchers & encourage partnerships between research organizations, advocates, & decision makers. It will increase its visibility & credibility among the donor community & will convince them of the utility of supporting HIV/AIDS prevention.

40. Summary Vision ✓ About the future ✓ Observable ✓ Idealistic ✓ Not about the program Mission ✓ Feasible ✓ Identifies activities and relationships ✓ About the program

41. Exercise 1: Correcting Vision and Mission Statements

42. Step 3: Boundary Partners

43. Boundary Partners are... Those individuals, groups, & organizations with whom a program: interacts directly to effect change can anticipate some opportunities for influence engages in mutual learning

44. Who are your boundary partners? Programme BeneficiariesStakeholdersBoundary Partners

45. DFID example… AcT Tanzania KPMG Network member National media Government department Community leaders District official Illegal loggers CSO CSO (TFCG) CSO

46. Strategic partners Selected for their potential to contribute to the mission A person or group with whom the program works directly to achieve the mission, without necessarily wanting to change the partner’s behaviour as part of the mission E.g. Donor agency, contracted service providers, NGOs doing similar work, media agencies

47. Facilitation questions In which individuals, groups, or organizations is your program trying to encourage change as a contribution to the vision? With whom will you work directly? Are you choosing boundary partners because you want to influence the ways they help or influence others? On whose actions does your success depend?

48. Exercise 2: Are you my boundary partner?

49. Swayamsiddha Mission Statement The Swayamsiddha Project works with governments, NGOs & CBOs to improve women’s health and empowerment. It facilitates the development of women’s self help groups. It provides them with funding and training to help them influence community and government services to be more responsive to their health and livelihood needs. It fosters mutual respect and joint action between these self-help groups and: banks; police; health and social service providers; and government agencies. It researches and promotes the application of ecosystem approaches to human health in agriculture and in the provision of health and sanitation services. Swayamsiddha addresses equity issues in all its activities. It uses participatory methods to monitor progress, to learn how to become more effective in supporting its partners and to report on its results.

51. Step 5: Progress Markers

53. (Deep transformation) (Active engagement) (Early positive responses) Love to see Like to see Expect to see Progress Markers

54. Love to see Like to see Expect to see Like to see Expect to see Like to see Expect to see

55. Why Graduated Progress Markers? Articulate the complexity of the change process Allow negotiation of expectations between the program and its partners Permit early assessment of progress Encourage the program to seek the most profound transformation possible Help identify mid-course improvements

56. ? Greater awareness… Empowered women… Community ownership… Reduced conflict… Increased collaboration… Governmental commitment… Gender sensitivity… Equal access… Budgetary transparency… Active participation… Poverty alleviation… Strengthened capacity… How can we measure:

57. Examples Project expects to see Tanzania Forest Service – Consult with communities on the new forest policy and the national REDD strategy; – Develop management plans for all Nature Reserves; – Continue a policy that is supportive of participatory forest management.

58. Examples Project would like to see Tanzania Forest Service – respond promptly to information from communities about illegal activities and other forest management issues and take action to resolve the issue. – incorporate recommendations from communities and civil society organistions in the revised forest policy and the national REDD strategy; – approve voluntary social and environmental timber standards; – implement and monitor management plans for nature reserves and other high biodiversity forest reserves with the participation of communities; – approve independent forest monitoring and cooperate fully in its implementation; – incorporated Biodiversity indicators in NAFORMA

59. Examples Project would love to see Tanzania Forest Service – successfully advocate for wider application of good timber standards throughout Tz gov’t systems. – consistently and effectively apply the Forest Act and regulations within the timber and charcoal trade thereby increasing the revenue that they capture and reducing illegal timber harvesting and charcoal production; – incorporated social and environmental standards in revised harvesting regulations and other documents related to the national forest programme; – Applying good social and environmental practices in respect of reserve management

60. Facilitation questions How can the programme know the boundary partner is moving toward the outcome? What would they be doing? What milestones would be reached as the boundary partner moves towards their intended role in contributing to the vision?

61. Progress Marker Checklist Each Progress Marker: Describes a changed behaviour by the boundary partner Can be monitored & observed As a set, Progress Markers: Are graduated from preliminary to more profound changes in behaviour Describe the change process of a single boundary partner

62. Exercise 4: Arranging Progress Markers

63. Step 4: Outcome Challenge

64. Outcome Challenge Describes behaviour of a single boundary partner Sets out the ideal actions, relationships activities Describes the boundary partner’s contribution to the vision

65. Facilitation questions Ideally, in order to contribute to the vision, what would the boundary partner be doing? With whom would they be interacting? How could this boundary partner contribute maximally to the vision?

66. Women’s self help groups are taking action to make community and government services more responsive to the health and livelihood needs of women and girls. They influence banks, police, health and social service providers, local officials and state and national government agencies in relationships of mutual respect and joint action to improve women’s well being. Women’s self help groups arrange bank loans for members and for life skills training for girls to be included in the school curriculum. They influence local, state and national government policies and expenditures on community improvement and transportation and support women candidates to run for election to local government office.

67. Step 6: Strategy Maps

68. Strategy Maps For each Boundary partner: What will the Project do to SUPPORT desired changes in the Partner towards the Outcome Challenge? Diverse range (an array) of ACTIVITIES: -Aimed at Partner -Aimed at Partner’s environment -Cause, Persuade, Support

69. Strategy Map causalpersuasivesupportive Partner Environment

70. Strategy Map causalpersuasivesupportive I aimed at individual boundary partner Direct output strong influence arouse new thinking; build skills, capacity on-going support E aimed at boundary partner’s environment alter the physical, regulatory or information environment broad information dissemination; access to new info create / strengthen peer networks and partnerships

71. Facilitation Questions causalpersuasivesupportive Partner what will be done to produce immediate outputs? what will be done to build capacity? how will sustained support, guidance or mentoring be provided? Environment what will be done to alter the physical or policy environment? how will the media or publications be used? what networks or relationships will be established or utilized?

72. SM summary Outlines the programmes approach in working with the BPs Indicates the relative influence the programme is likely to have on the BPs Helps identify gaps in strategy – not to fill the boxes Suggest the type of evaluation methods needed

73. Exercise 5: Strategy Map

74. Step 7: Organisational Practices

75. Organizational Practices How does your team or organization stay relevant, viable and effective?

76. 8 practices 1.Prospecting for new ideas, opportunities, and resources 2.Seeking feedback from key informants 3.Obtaining the support of your next highest power 4.Assessing and (re)designing products, services, systems, and procedures

77. 8 practices 5.Checking up on those already served to add value 6.Sharing your best wisdom with the world 7.Experimenting to remain innovative 8.Engaging in organizational reflection

78. Exercise 6: Sharing organisational Practices

79. M&E planning and journals

87. M&E can be used for… Feeding learning into management cycle Meeting accountability / reporting requirements Reducing risk Multi-actor learning process Decision-making Inform publicity documents, communication activities, or case-study materials What else…?

88. Effective use of M&E depends on… Adjust Strategy Improve Implementation Report Performance Risk Management Build Capacity Lobbying Annual Staff Mtg Annual Report Quarterly Report to Donor 3yr Prog Review Proj team Mtg Program Staff Board Donor Project Partner Program Management Strategies implemented Morbidity & mortality partner actions in community community responses to proj. partner collab. with ministry community participation funding flows USESEVENTS USERS INFO

89. Effective use “I can honestly say that not a day goes by when we don’t use those evaluations in one way or another.”

90. 5 kinds of M&E Information Program Partner outcomes (behaviour changes in the partners) implementation (interventions by the program) relevance & viability (actions of the program) C o n t e x t u a l I n f o r m a t I o n State, status or situational data Strategies

91. Lets look at Swayemsida in a simple form… 1. Participating when abuse is discussed 2. Attending self help group meetings Like to see women victims of domestic violence: 3. Educating self about combating domestic violence 4. Setting an agenda for taking collective action 5. Raising funds for victim support services Love to see women victims of domestic violence: 6. Taking action to protect self 7. Lobbying police to change enforcement practices Expect to see women victims of domestic violence:

92. M&E Planning Worksheet What information will be collected? How will it be collected, from what sources and when? Who will collect it? Who will make sense of the info & how? Who will manage the collection-to- usage processes? What resources are needed & who contributes?

93. Focus your questions… many interesting & important questions, but data costs money & time vague questions yield vague answers avoid nice-to-know & might-as-well data OM can’t answer all questions but it can help illuminate: achievement of outcomes (expected & unexpected) factors and actors contributing to outcomes implementation of the program

94. Outcome Mapping M&E… systematic collection of data a regular learning & improvement cycle encourages the program to challenge itself framework for collecting & organizing, does not analyze data nor tell you how to use it

96. Outcome journal What, when, who, how much Contributing factors Sources of evidence Unanticipated change Lessons, required changes, action

97. Outcome journal

98. Strategy journal What did you do, with whom, when (How) did it influence change Outputs Suggested changes/follow up Lessons

99. Performance journal Progress against internal commitments – e.g. organisational practices Reviewing the logic of the programme

100. Journal for Monitoring Outcomes

101. Anecdote as evidence

102. Taking up M&E findings… What should we keep doing? What do we need to change in order to improve? What strategies/practices do we need to add? What strategies/practices do we need to drop? (i.e., they produced no results, they require too much efforts or resources to produce results?) Has any issue come up that we need to evaluate in greater depth? What? When? Why? How?

103. Considerations for internalizing OM… Decision / will (of the organization, of those involved) Incentives (learning) One person or team that is responsible (that has the support of governing bodies); facilitator; champion (at least one, more is better) Management-donor buy-in (challenges of that) Experiment or to meet needs

104. Coming back to indicators …

105. Don’t get lost in music … Appreciative inquiry Case study Composite Logic Model Content analysis Contribution analysis Cost-Benefit Analysis (CBA) Creative expression Critical Stories of Change Developmental Evaluation Do No Harm Dreams realised or visioning Episode Studies Expert review GIS mapping Graphing results Group assessment Historical trends and timelines Horizontal evaluation Impact evaluation Impact flow diagram Impact Pathway Innovation Histories Institutional history Institutional linkage diagram Learning alliances Learning-oriented evaluation Lot Quality Assurance Sampling Logical Framework Analysis M&E wheel (or "spider web") Method for Impact Assessment of Programmes and Projects (MAPP) Modular Matrices Monitoring of Effects (MOVIE) Most Significant Change (MSC) Network Functions Approach NGO-IDEAs Toolbox Non-random sampling Observation Outcome Mapping (OM) Participatory Impact Monitoring (PIM) Participatory Impact Pathways Analysis Participatory Livelihood Monitoring (PaLSA) Peace and Conflict Impact Assessment (PCIA) / Aid for Peace (AfP) Planning and Implementation Framework Analysis (PIFA) Poverty Impact Assessment (PIA) RAPID Outcome Assessment Results Based Management Rich pictures (or mind maps) Rigorous Impact Analysis (RIA) SAGE – Situational Analysis and Goal Establishment Social Framework Social mapping or well-being ranking Social Network Analysis Sociograms Stakeholder analysis Sustainability Impact Assessment (SIA) SWOT Systems (or inputs-outputs) diagram Theory of Change (ToC) Theory-based evaluation Transects

106. Exercise 7: Pros and cons of OM

107. Thank you! For more info on OM: Visit the OM learning community: www.outcomemapping.ca Contact: Simon Hearn, s.hearn@odi.org.uk Julius Nyangaga, julesnyangaga@yahoo.co.uks.hearn@odi.org.ukjulesnyangaga@yahoo.co.uk Ziad Moussa, moussa.ziad@gmail.commoussa.ziad@gmail.com

108. outcomemapping.ca