A Critical Analysis of Current M&E Frameworks Insights from AWID’s research on M&E frameworks By Srilatha Batliwala, Scholar Associate, AWID SBatliwala, AWID December, 2009

Draft for Discussion AWID’s research on M&E is being broadened, deepened and sharpened, and a newer version of our overview paper will be available in the second half of 2010 This PowerPoint is thus a reflection of our work-in-progress, and intended for discussion, rather than a final viewpoint. SBatliwala, AWID

First, let’s remember that: “When you work for women’s interests, it’s two steps forward - and at least one step back. And those steps back are… often evidence of your effectiveness; they represent the threat you have posed to the power structure, and its attempt to push you back. Sadly, even our ‘success stories’ are sometimes nothing more than ways the power structure is trying to accommodate and contain the threat of more fundamental change by making small concessions to us.” Sheela Patel, 1987 SBatliwala, AWID

Assumptions in M&E 1.Everything should be measured 2.Everything can be measured – even complex processes of social change 3.Measurement will enhance our ability to accelerate, deepen, replicate positive change, or improve achievement of desired goals 4.Change is predictable – we know what it will look like, where it will occur, and how to assess it SBatliwala, AWID

5.The macro-political environment is stable, democratic, upholds basic rights, and protects change agents and change processes – i.e., law and order, an impartial judiciary and police, due process, rights of association, civil liberties, an independent media, etc. are inevitably present in every context 6.The organizational environment is stable and unchanging – staff don’t leave, core operational costs are secure, and you haven’t been shut down or harassed Assumptions in M&E SBatliwala, AWID

Why M&E? In theory, to: Learn how change happens, make it happen faster, better, etc. Analyze and sharpen our role in the change process Empower our constituencies Practice accountability – to donors, constituencies, peers, public, etc. To advance our advocacy agenda SBatliwala, AWID

Why M&E? In practice, to: Satisfy donor requirements Prove we’re doing what they funded us to do Leverage more funding Protect ourselves - from backlash, attacks, slander campaigns, etc. SBatliwala, AWID

What isn’t working Very few M&E frameworks actually enable us to understand whether change has happened, or gender power altered – they measure performance (program inputs and outputs), not change What they measure may not be indicative of the change they’re measuring at all – (e.g., GDP!) SBatliwala, AWID

What isn’t working Many current frameworks are very linear, and look for simplistic cause- effect relationships (x intervention = y effect = z change) One such, the logical framework, is often focused on performance - “[log frames] focus on the expected achievements laid out in the matrix – rather than the work itself.”, SBatliwala, AWID

What isn’t working Most frameworks do not provide for tracking negative change, reversals, backlash, unexpected events, etc., that push back or shift the direction of the change process. In women’s rights work, this is vital, because the most effective work seriously challenges patriarchal / other social power structures, creating negative reactions. Reactions / backlash / negative change is often evidence of positive impact! SBatliwala, AWID

What isn’t working Most current frameworks are completely inappropriate for certain kinds of social change organizations and strategies: E.g., advocacy, training / capacity building, knowledge production, challenging discourse, etc. Consequently, these organizations are forced to measure their processes, outreach and outputs ( number programs held, number of participants, publications, attendance at rallies, etc. ), rather than their impact. SBatliwala, AWID

What isn’t working Several false binaries and dichotomies are embedded within or underlie many M&E approaches – e.g., “quantitative- qualitative”, “subjective-objective”, “macro-micro”, “success-failure”, and so forth. These create problematic hierarchies rather than approaches that can integrate and transcend such dualities. SBatliwala, AWID

What isn’t working Disjuncture between change measures and our time frames. The changes we are trying to track may not be visible within the time frame in which we are seeking them Most current M&E frameworks are neither gendered nor feminist in their principles or methodology. “It takes ten years to build an organization, twenty years to build a movement, and thirty years before you see lasting impact.” Ela Bhat SBatliwala, AWID

Measuring Social Abstractions – the key questions 1.What is it? Definition 2.Where is it? Location (geographic, social, political, institutional, sectoral) 3.What are its boundaries? Spatial, demographic, and conceptual 4.What does it look like? Characteristics & situational analysis 5.What can be measured? Measurable dimensions, sub-units, indicators 6.When do we measure? Baseline + frequency of assessment SBatliwala, AWID

Elements of Feminist M&E Feminist answers to the social abstraction questions Informed by / embedded in feminist values and principles Based on complexity Use the best available tools (both quantitative and qualitative) Appropriate and differentiated time frames Track backlash / reversals Designed for the level, nature of work and strategies of each organization SBatliwala, AWID

Principles of Feminist M&E 1.The right of our constituency to inform, co- design and participate in the monitoring and evaluation of change processes 2.Respect for the voice and perspective of all key stakeholders 3.Prioritizing learning in our M&E goals 4.Positioning M&E as a political activity 5.Integrating political and social forces into our analysis & frameworks SBatliwala, AWID

Principles of Feminist M&E 6.Avoiding attribution, assessing contribution 7.Eschewing false binaries / dichotomies 8.Not using M&E for punitive purposes 9.Capturing, analyzing and addressing negative changes, reversals 10.Willingness to abandon, revise, recast our frameworks SBatliwala, AWID

First steps in doing it differently: 1.Problem definition & situational analysis 2.Enunciation of our M&E principles 3.Development of customized M&E framework & indicators (see forthcoming AWID guideline) 4.Establish a baseline 5.Track and map Our processes Our performance and External change – positive and negative SBatliwala, AWID