1. An Introduction to Outcome Mapping Tools for Planning, Monitoring & Evaluating Development Projects and Programs IPDET June 16, 2011 Terry Smutylo tsmutylo@magma.ca

2. 2 Objectives for this Session 1.Inspire you to search for and adapt evaluation tools to fit your context & needs 2.Introduce key concepts & tools in Outcome Mapping

3. 3 Development interventions can be: Complex (involve a confluence of actors and factors) Unstable (independent of project duration) Non-linear (unexpected, emergent, discontinuous) Two-way (intervention may change) Beyond control (but subject to influence) Incremental, cumulative (watersheds & tipping points)

4. 4

5. 5 Challenges in evaluating development interventions: establishing cause & effect in open systems measuring what did not happen reporting on emerging objectives timing: success today, failure tomorrow? encouraging ongoing learning reconciling differing values, perspectives measuring sustainable results

6. 6 program influence decreases (is replaced) changed behavior The focus of Outcome Mapping community capacity & ownership increases

7. 7 7 What is Outcome Mapping? A project/program management tool for: Clarifying intentions Developing strategies to achieve results Identifying information for monitoring and evaluating Fostering organizational learning Enhancing other frameworks & methods

8. 8 1990s: IDRCs post-Rio search for sustainable results 1998: Barry Kibel and Outcome Engineering 1999: Collaboration & testing within projects 2000: Publication of manual in English & French 2002: Training, facilitation & usage globally 2006: OM Learning Community: www.outcomemapping.ca www.outcomemapping.ca 2008: CLAMA OM: brief history

9. 9 Three key concepts in OM: Sphere of influence Recognizing changes in behaviour as outcomes Boundary Partners

10. 10 Beneficiaries Project or Program Partners There are limits to our influence sphere of influence sphere of concern

11. 11 inputs activities outputs Changes in behaviour as outcomes sphere of influence sphere of concern o u t c o m e s changes in conditions, well-being changes in behaviour

12. 12

13. 13 improved human, social, & environmental wellbeing Step 1:

14. 14 I have a dream! Martin Luther King, Jr. August 28, 1963

15. 15 The mission is that bite of the vision statement on which the program is going to focus. Mission Step 2:

16. 16 A mission statement describes: What do you do? Who are your principle collaborators? How do you work with them?

17. 17 about the future concrete, observable idealistic not about the program feasible identifies activities and relationships about the program VisionMission

18. 18 Step 3: Who are our Boundary Partners? Boundary Partners Beneficiaries other stakeholders Project or Program

19. 19 Participatory research on demonstration farms to develop approaches to drip irrigation Extension workers visit demonstration farms Training of extension workers Publication of performance of different set- ups Extension workers explain & promote drip irrigation Farmers adopt drip irrigation methods Greater quantities of groundwater available Farmers participate in field trials using drip irrigation Farmers add to own knowledge of techniques Reduced numbers of new wells Adapted from K. Kelpin, 2009

20. 20 Progress Markers (Deep transformation) (Active engagement) (Early positive responses) Love to see Like to see Expect to see Step 5:

21. 21 Why use Progress Markers? Articulate the complexity of the change process Encourage the program to seek the most profound transformation possible Facilitate negotiation of expectations with partners Enable early assessment of progress Help identify mid-course improvements

22. 22 Progress Markers – IMF examples Expect to see local communities: Participating in regular model forest (MF) meetings Establishing an organizational structure for cooperation Acquiring new skills for managing model forests Articulating a locally relevant vision for the MF Like to see local communities: 5.Contributing resources to set up their MF 6.Calling upon external experts for advice 7.Seeking out new partners for the MF Love to see local communities: 8.Obtaining funding from different national sources 9.Publishing examples of benefits achieved through MF 10.Helping other communities establish MFs 11.Sharing lessons learned internationally 12. Influencing national policy debates on resource use

23. 23 Sample progress markers Like to see Womens Self-Help Groups: 4.Soliciting training in maternal & child health for its members 5.Acquiring skills in managing credit programs 6.Lending money to members Love to see Womens Self-Help Groups: 7.Lobbying local government for expenditures on community improvements 8.Putting forth candidates for election to local government council Expect to see Womens Self-Help Groups: 1.Holding regular meetings 2.Discussing a list of shared concerns 3.Contributing to a group bank account

24. 24 6 kinds of strategies CausalPersuasiveSupportive I aimed at Individual boundary partner Strong, direct influence Arouse new thinking; build skills, capacity Continuing support E aimed at boundary partners Environment Alter the physical, regulatory or information environment Broad information dissemination; Access to new info Create / strengthen peer networks

25. 25 causalpersuasivesupportive I E Step 6: Strategy Map

26. 26 Step 7: Organizational Practices How does your team or organization stay relevant, viable and effective?

27. 27 You: keep learning foster creativity & innovation seek better ways to assist your partners maintain your niche maintain high level support build relationships

28. 28 Organizational Practices 1.Prospecting for new ideas, opportunities, and resources 2.Seeking feedback from knowledgeable informants 3.Maintaining the support of your next highest power 4.Assessing and redesigning products, services, systems, and procedures

29. 29 …organizational practices 5. Getting feedback from those already served 6. Sharing your learning with the world 7. Experimenting to remain innovative 8. Engaging in organizational reflection

30. 30

31. 31 5 kinds of M&E Information Program Partner outcomes (behaviour changes in the partners) implementation (interventions by the program) relevance & viability (actions of the program) C o n t e x t u a l I n f o r m a t I o n State, status or situational data Strategies

32. 32 Changes in well-being Changes in behavior Capacity includes the power & responsibility to act Need for diversified strategies Influence not control Operating Principles of OM: Contribution not attribution

33. 33 OM is designed to be: Flexible: modular to be adapted to use & context Complementary: can be combined with other methods Evaluative: promote a culture of reflection, and social & organizational learning Participatory: seek dialogue and collaboration with partners

34. 34 1.There is no silver bullet 2. Seek quality with passion & integrity 3. Recognize & celebrate achievements of your partners 4. Be idealistic realists. 5. Learn, and teach upwards. In monitoring & evaluation:

35. 35 Inspiration & Information Visit the OM learning community: www.outcomemapping.ca www.mapeodealcances.net http://publicwebsite.idrc.ca/EN/Themes/Evaluation/Pages /default.aspx