Presentation on theme: "Comprehensive M&E Systems"— Presentation transcript:
1 Comprehensive M&E Systems Identifying Resources to Support Monitoring & Evaluation Plans for National TB Programs
2 Objectives of SessionIdentify the key elements of a comprehensive M&E systemIdentify importance and objectives of an M&E systemDescribe key steps to strengthening or establishing and M&E system
3 Key Elements of an M&E System M&E managementLinked to program/strategy goals and objectivesIndicatorsData collection systemsData analysis, use & dissemination plans
4 Information Continuum Show a continuum of data demand, generation, collection and useactivities. This continuum is cyclical, reflecting an ongoing, iterative process. Partners work to ensure that data users demand quality data and that data producers understand users’priorities and information needs.
5 How can a comprehensive M&E system support the NTP? Efficient use of data and resourcesResponsive to key stakeholdersGuarantee that donor funded M&E efforts contribute to national needsEncourage communication between all those responsible for addressing TBWith strong M&E system you can ensure that indicators and methodologies are comparable over time and reduce duplication of effort. Particularly important where resources are scarce.NTP often need to report to various constituents (program managers, politicians, donors, researchers, etc…)Rather than simply serving the reporting needs of agencies or legislatures overseasLab, nurses, community outreach, etc… requires shared planning, coordination execution and analysis, or data dissemination
6 Objectives of an M&E system Measure progressImprove accountability and management of resourcesEfficiently and effectively use dataImprove coordination with partnersCollect complete and timely information on national effort to control tuberculosis
7 Steps to strengthen and implement an M&E System 1.) M&E Situation Analysis2.) Develop M&E plan3.) Establish/Strengthen M&E Unit4.) Implementation of M&E Plan5.) Mechanisms for Management and Quality Control of M&E ActivitiesWe have already done 1. and 2. and this presentation and following group activity will focus on 3 and 4.
8 Step 1: M&E Situation Analysis Objectives:Identify existing M&E practices, indicators, and data sources already being collectedIdentify gaps where indicators will need to be developed and data sources identified for the M&E plan of the national TB programAssess national M&E capacity and needs for capacity developmentIdentify potential for integration of TB M&E into national HISReview this quickly
9 Step 1: M&E Situation Analysis MethodologyDesk review: essential documentsStructured interviews with national level stakeholdersStructured interviews with regional and district level stakeholders, including community organizationsStructured interviews with data users at all levelsNeeded?
10 Step 2: Develop M&E planDevelop clear goals and objectives to meet challengesPropose interventionsSelect and/or develop indicators to measure interventionsIdentify data sources and ensure collection is feasibleIdentify those responsible for data analysis, use and disseminationUsing the M&E framework…This is what we did this morning
11 Step 3: Establish/Strengthen M&E Unit Unit within the NTP or implementing organizationEnsure adequate financial resourcesStrengthen human capacityEnsure regular supervision of data collection and reportingFinancial resources- Budget of 5-7% of grant proposal- If within NTP, significant government contributionHuman resources- M&E expertise leading and involved in the unit (program monitoring, information systems, epidemiology, statistics, data processing, and social science)- Involvement of academia (epidemiology, social sciences, communication)Involvement of key NGOs, donors and international organizations*It is essential to work within the existing system and unit
12 Step 4: Implementation of M&E Plan Develop strategy to build and maintain M&E capacity at national and sub-national levelsAdapt forms and guidelines for M&E at different levelsCoordinate with sub-national, national and international M&E experts to support implementation of planTraining of staff at different levelsDevelop guidelines for reporting requirements
13 Step 5: Mechanisms for Management and Quality Control of M&E Activities If necessary, adapt national data base for indicators and report generationCarry out regular assessments of quality of M&E reporting at district (and regional) level and sectors (bi-annual system, each year half of the districts)Carry out regular assessments of quality of program monitoringEstablish feedback mechanisms!
14 M&E System CostsRule of Thumb is 10% of program budget (GFATM says 5-7%)- Situation Analysis and Design of M&E plan for proposal- Strengthening of M&E unit- Routine Data Collection- Data Management and Reporting- Build capacity for analysis and use of data- Quality Control Improvements- Special surveys
15 Priorities for M&E Systems Development and Strengthening - Who will implement the system?- How will the data be collected?- How will the data be analyzed and used?- How will the quality be ensured?If there is good M&E plan in the proposal, it should be able to answer all of these questions.
16 Checklist of Good M&E System M&E UNITA unit or individual within the central unit of the program who is responsible for M&EA significant contribution to the national M&E budgetA formalized (M&E) link with national and local research institutions, professional associations, and academic institutionsA formalized (M&E) link with leading NGOs, donors, and community-based organizationsEpidemiologist and/or social science expertise in the M&E unit or affiliated with the unitData processing and statistical expertise in the M&E unit or affiliated with the unitData dissemination expertise in the M&E unit or affiliated with the unitClear GoalsWell-defined national program goals and targetsRegular reviews/evaluations of the progress of the implementation of the national program plansGuidelines and guidance to districts and regions or provinces for M&EGuidelines for linking M&E to other sectorsCoordination of national and donor M&E needsIndicatorsA set of priority indicators and additional indicators at different levels of M&E, some of which are comparable over time and with other countriesData Collection, Analysis, and UseAn overall national data collection and analysis planA plan to collect data and analyze indicators at different levels of M&ESystemwide knowledge and capacity (e.g., tools and budget) to collect and use dataA plan to supervise, support, and ensure the quality of data collectionA plan to ensure the translation of data into problem identification, strategic planning, policy formulation, etc.Data DisseminationAn overall national data dissemination planA well-disseminated informative annual report of the M&E unitAnnual meeting to disseminate and discuss M&E and research findings with policy-makers and plannersA clearinghouse for generation and dissemination of findingsA centralized database or library of all data collection, including ongoing researchCoordination of national and donor M&E dissemination needsRun through this and pass out handout of this. Explain that the next group activity will be using this as a point of discussion to asses own M&E system
17 Lessons learned from successful M&E Systems Collect complete input and output dataRequires internal self-assessment and external evaluationSystem should be a simple as possibleStandardized core set of tools to collect and analyze dataHuman capacity to collect, verify, enter and analyze dataM&E system built into the design of the programSub-national data important for national level and even more so for program management and decision makingSource: GFATM Tool Kit
18 ResourcesGFATM (2004). Monitoring and Evaluation Toolkit HIV/AIDS, Tuberculosis and MalariaWHO (2004). Global Tuberculosis Control: Surveillance, Planning, Financing. WHO Report 2004.WHO/IUATLD (2005). Anti-TB Drug Resistance Surveillance in the World. Report No. 3.WHO (2004). Compendium of Indicators for Monitoring and Evaluating National Tuberculosis Programs.WHO (1998). Tuberculosis Handbook.SWEF Research Network (2003). Common Research Protocol. Monitoring and Evaluating the Health System-wide Effects of the Global Fund to Fight AIDS, Tuberculosis and Malaria.PHRPlus (2003). The System Wide Effects of The Global Fund : A Conceptual FrameworkEcon Analysis (2005). Inventory of M&E practices and systems for global health organizations.