An Introduction to an Integrated P,M&E System developed by IDRC Kaia Ambrose October, 2005

Outline A little context: my perspective OM and IDRC Key aspects of OM Stages of OM A few lessons from the field

My perspective The Ceja Andina Project Ecopar Ecuadorian NGO - research, training and capacity building for natural resource conservation The Ceja Andina Project Sustainable use of agriculture and forest biodiversity in the ceja andina (cloud forest) region of the northern Ecuadorian Andes. Research-oriented (traditional and participatory), social learning processes, policy development and strengthening of local government.

Planning, Monitoring & Evaluation concerns in the Ceja Andina Project How can we make learning & action on-going, iterative? How can we (as an organization) learn from the project in order to improve the way we intervene? How can we identify our contributions to development “along the way” and not just downstream? How can we bring creativity and interest to the PM&E process? How can PM&E be a participatory process conducive to social learning and good decision making?

Origin of OM Focusing on & measuring downstream impact Created by the International Development Research Centre (IDRC) between 1998 and 2000 as a response to difficulties faced by their programs and projects Endogenous decision-making processes through PM&E “Measuring” the unmeasurables Necesidad de: enfatizar en aspectos que permitan la sostenibilidad del desarrollo (i.e.“capacidad para el desarrollo”) capturar información útil, específica al ámbito de influencia de los programas

What is Outcome Mapping? An integrated planning, monitoring & evaluation approach An approach that defines outcomes as changes in the behaviour, relationships and actions of stakeholders / partners An approach for designing in relation to the broader development context but assessing within your sphere of influence A methodology that characterizes and asseses the program’s contributions to the achievement of outcomes

OM Key aspects Development occurs in open, complex and dynamic systems Development changes are determined by multiple causes, factors and actors Recognizes multiple logics, non-linear relationships, uncertainty, virtuous and vicious cycles in development processes Considers relationships and influences among the partners Attribution of results is not always possible Importance of change as an ongoing process The Program as an organizational unit is an “agent of change” and “subject to change” Sistemas abiertos y complejos: en donde se reconocen multiples logicas relaciones no lineales, incertidumbre, circulos virtuosos y viciosos Definir Causa y Efecto en un Sistema Abierto We work in complex, open systems (many things happen at once, we aren`t the only people involved, relationships among events and actors creates a changing contex) Linear logics models (if/then, so what?) aren`t adequate to capture this Emphasis on attribution usually tries to credit a single actor whereas, in reality, multiple actors create results and need credit Focus on positive intended results is insufficient as unexpected positive and negative results occur and should be accounted for and learned from We were faced with the challenge of being honest & realistic in crediting the multiple actors & factors that contribute to development Is this a problem you are grappling with? How?

Key differences Key differences with conventional PM&E approaches: Linear, cause-effect, thinking contradicts the understanding of development as a complex process that occurs in open systems So ... No claim attribution, just assessing contribution by those with whom programme works directly Focus on changes in behaviours and relationships, not just actions

Assessing Development Results (in this case, Outcomes) Behaviour Changes

Assessing Internal Performance Behaviour Changes Program

Assessing Influence Behaviour Changes Program

Program’s Sphere of Influence The World Beyond Program = Program`s Boundary Partners

OM as a process Encourages a “culture of reflection and evaluation” in all stakeholders’ involved Enables social learning Supports empowerment and self determination through participation and iterative learning Facilitates more equitable dialogue processess Facilitates organizational learning Decision making and action

Intentional design: why, who, what and how

Characterizing and Assessing Outcomes Program BoundaryPartner Strategy Journal Performance Journal Outcome Journal Organizational Practices Strategy Map Outcome Challenges & Progress Markers

Vision Ceja Andina Project Ecopar Mission BP2 BP3 BP1 BP4 To enable the sustainable use and restoration of biodiversity and agrobiodiversity in the natural and cultural landscape of the ceja andina Vision Mission BP2 Rural Schools BP3 Municipalities BP1 Farmer Groups BP4 Local NGOs Outcome Challenge 1 and PMs Outcome Challenge 2 and PMs Outcome Challenge 3 and PMs Outcome Challenge 4 and PMs Strategy Maps Strategy Maps Strategy Maps Strategy Maps Ceja Andina Project Ecopar (organizational Practices)

Aspects that can be improved Situation and stakeholder analysis explicitly incorporating questions to enable us to articulate a social change theory Identify a set of complementary tools that can be applied in each of the OM steps according to the users needs Facilitation questions to define M&E priorities Deepen the principles of Utilization Focused Evaluation More attention to risk understanding and management (i.e. risks against powerful political/economic forces)

http://www.idrc.ca/evaluation