Community Learning Systems : Increasing Accountability for Results from Development Assistance Joy P. Mukaire Lawrence L. Jackson Annual Conference 2009 Christian Connections for International Health

The Shift in Accountability  Change to a transitional phase should be reflected in the shift from the traditional role of NGOs in health of service provision  From little accountability to anyone other than the individual NGO and their donors  To a change demanding that NGOs implement health care on behalf of government, working alongside MoH GOSS to support national frameworks, supporting MOH in health systems strengthening at all levels.

Struggles of M&E Systems  Harmonization of M&E approaches  Having one National Health Strategic Framework  Having one National M&E framework  Having one National Coordinating body  Strengthening and expanding M&E Capacity at all levels  Promoting decentralization of M&E  Improving human capacity for M&E  Allocating adequate resources to M&E

Struggles of M&E Systems  Using national data to “prescribe solutions” for local level problems/needs  Treating every community/district the same even when priorities differ in nature  Not knowing the magnitude of the problem being addressed  Mismatch between national priority and local needs  Promoting utilization of program data  To decide on priorities  Make tactical changes to improve programs  To scale up proven project models

Management for Results There is increasing pressure to manage programs with focus on achieving results that are visible. Monitoring and Evaluation has two purposes:  Aid program managers to improve their programs by using data  Sub-national level (Implementation Teams)  National level (Policy and Sr. Management)  Support a reporting function  National stakeholders  Internationally [Donors, International Organizations (RBM)]

Macro vs Local M&E Outcome Data NATIONAL LEVEL Program District 1 Program District 2 Local managers need M&E data that show what is happening in the areas where they work

Efforts to Strengthen M&E  Rapid Assessment using Lot Quality Assurance Sampling  Sampling method used in industry since the 1920s  Adapted to public health settings in mid-1980s  Local managers at a sub-district level using small samples determine whether a performance target has been reached  Yes or No Judgment

What is LQAS? An analysis method that can:  be used locally – in “supervision areas”  to identify priority areas: those not reaching an established performance benchmark for an indicator  data for local management decision making and for sharing information across supervision areas  measure coverage at an aggregate level (e.g., program catchment area or district or nation)  be used locally – in “supervision areas”  suitable for Reporting Purposes  LQAS uses small samples (less than 20)

A B C D E F G Assume a program area that has 7 supervision areas or municipalities Each one is supervised by one person Each one has between promoters/wards /communities to supervise

A B C D E F G Good Below Average or Established Benchmark

Good Below Average or Established Benchmark Identify the reasons for program problems Develop targeted solutions Maintain the program at the current level Identify Supervisors and Health Workers that can help other Health Workers improve their performance

These are the problem supervision areas. The Decision Rule is 12 women delivering with a Trained Provider

District S.A. = 19 District S.A. = 19 District S.A. = 19 Nation or Region SA= Supervision Area

LQAS Application in CHAS Applied in two counties of Leer and Koch in Unity State to establish baseline for a CHAS implemented Access to Basic Service “Quick Impact Project”  Estimate immunization coverage among children aged 12 – 59 months;  Assess knowledge and practice of correct feeding practices among pregnant women and children aged 0 – 23 months;  Estimate coverage of ITNs among children under 5 and pregnant mothers;  Estimate prevalence of ARI, fever, and diarrhea among children under 5;

LQAS Application in CHAS  Assess knowledge and practice of mothers and caretakers of children under 5 of critical danger signs of and correct response to ARI, fever and diarrhea;  Utilization of ANC, delivery and PNC services among pregnant women; and  Knowledge and use of family planning methods among women in the reproductive age bracket.  Use of sanitary means of excreta disposal  Use of improved water source of drinking water

LQAS Application in CHAS The sample: A total of 5 parallel samples corresponding to the 5 project target groups, namely,  Heads of household  Mothers of children 0 – 11 months  Mothers of children 12 – 23 months  Mothers of children 0 – 59 months  Women aged 15 – 49 From each randomly selected village  133 households in Koch x 5 interviews =  152 households in Leer x 5 interview =

LQAS – A Catalyst for Change A team of 14 people were trained in Unity State, six at HQ and 8 at county level  Mobilizes managers and their organizations to appraise their programs  Empowers them by creating a learning environment  Supports leadership at national, district and sub-district levels

Advantages of Using LQAS  Can be used at a local level with modest amounts of supervision  Produces information that can be rapidly interpreted by local managers  Identifies where the successes and challenges are located  Paper/pencil analyses rather than requiring computer analyses  Data can be used for national reporting, and  Sampling theory is rigorous

Lessons Learned  LQAS findings revealed substantial variations in estimates of counties  The magnitude of “Sanitation” problem became more visible to implementers than before something the project was not even planning to address.  The substantial variation imply that local managers are better placed to make tactical changes in their Programme  Those responsible for providing services are better placed to analyze the challenges in their areas and decide on the strategic change

The Christian Perspective  This is one solutions to increased STEWARDSHIP of faith-based organizations  This tool is just but a means to acceptable standards of ACCOUNTABILITY where resources are limited and competing priorities are many  This tool builds a culture of result-oriented management and the culture of accountability, a contribution Christian health services can make to Strengthening Health Information Systems and Monitoring and Evaluation

The Power of Measuring Results  If you don’t measure results you cant tell success from failure  If you don’t see success, you cannot reward it  If you cannot reward success, you are probably awarding failure  If you cannot recognize failure, you cannot correct it  If you cannot demonstrate results, you cannot win public support Source: Adapted from Osborne & Gaebler 1992

Getting Ready

Household Interviews