Presentation on theme: "Unit 10. Monitoring and evaluation"— Presentation transcript:
1 Unit 10. Monitoring and evaluation Photo from:WHO. Revised TB recording and reporting forms, Version WHO/HTM/TB/TB Infection Control Training for Managers at the National and Subnational Levels
2 Objectives After this unit, the participant will be able to: Define monitoring and how it is essential to supervisionList two indicators to use in monitoring infection control at the national level, and two for use at the facility level[Review slide]
3 Outline Definitions: monitoring vs. evaluation Types of indicators: input, output, outcome and impactChoice of indicatorsExamples of indicators at global, national and facility levelsSetting targets[Review slide]
4 Develop the plan (to include goals and objectives) Lifecycle of IC planDevelop the plan (to include goals and objectives)ImplementThis is the same diagram we discussed in the Managerial unit.Like other good plans, the IC plan has goals and objectives.As a plan is being implemented, it will need to be fine-tuned as the situation changes.Monitoring helps the manager decide what revisions are needed in the implementation, so objectives are met.Monitor,evaluateRevise
5 Monitoring is….The routine tracking and reporting of priority information about a programme or project, its inputs and intended outputs, outcomes, and impact[Review slide]We’ll define inputs, outputs, outcomes and impact in a few minutes.Citations:WHO. A guide to monitoring and evaluation for collaborative TB/HIV activities WHO/HTM/TB/ , WHO/HTM/HIV/09.01Global Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit-2009 version.
6 MonitoringUses data obtained through record keeping, regular reporting, surveillance systems, observation or surveysHelps managers determine which areas require greater effortIs essential for supervisionIs also used to measure trends over time, so methods need to be consistent to ensure appropriate comparison[Review slide]Citations:WHO. A guide to monitoring and evaluation for collaborative TB/HIV activities WHO/HTM/TB/ , WHO/HTM/HIV/09.01Global Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit, 2009 version.
7 Evaluation is…The rigorous, scientifically-based collection of information about the program activities, characteristics, and outcomes that determine the merit or worth of the programme[Review slide]Citation:WHO. A guide to monitoring and evaluation for collaborative TB/HIV activities WHO/HTM/TB/ , WHO/HTM/HIV/09.01Global Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit, 2009 version.
8 Evaluation studiesProvide credible information for use in improving programmes, identifying lessons learned, and information decisions about future resource allocation[Review slide][Ask participants:]Can you name some ways that monitoring and evaluation are different?[Possible responses]:The main purpose of monitoring is to help managers routinely monitor progress and make needed corrections during programme implementation.By contrast, evaluations are done to determine the worth of a program.While monitoring is regular and ongoing, Evaluation studies are done less frequently.They also use different methods. Monitoring uses routinely collected information.Evaluation requires a special study (although it should leverage any data or surveys are are regularly undertaken).Citations:WHO. A guide to monitoring and evaluation for collaborative TB/HIV activities WHO/HTM/TB/ , WHO/HTM/HIV/09.01Global Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit, 2009 version.
9 M&E framework long term, cumulative effect of the programme over time Impactlong term, cumulative effect of the programme over timeOutcomeshort and medium term effects of the outputsOutputproducts or deliverables of the programme’s interventionsWe’ll work our way from the bottom to the top of this framework.For a programme to achieve its goals and objectives, inputs (financial, human and material resources) must result in outputs (products or deliverables)If the outputs are achieved, the programme will have a positive outcome, which is a short and medium term effect.These outcomes should have a long term impact.In this monitoring and evaluation (M&E) framework, the blue boxes are linked (yellow arrows) in a chain of results.Citations:WHO. A guide to monitoring and evaluation for collaborative TB/HIV activities WHO/HTM/TB/ , WHO/HTM/HIV/09.01Global Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit, 2009 version.Diagram adapted from: Dominique MathiotInputfinancial, human and material resources
10 Example of a results chain ImpactFewer health care workers with TBOutcomeCoughing patients are rapidly identified, separated, and servedOutputHIV clinics have written triage procedures for coughing patientsOur example is a country which has identified HIV clinics as high risk settings for TB transmission.Let’s work our way from top to bottom of the diagram; we’ll follow the chain of results first using the white boxes.One of the overall goals of the country’s IC plan is to decrease the spread of TB in order to protect health care workers and other patients.To have this impact, the country wants to be sure that coughing patients are rapidly identified, separated, and served.This will require written procedures be in place for coughing patients, which in turn requires there be an IC focal person identified and trained in each facility.We can check our logic by working our way back up the results chain by following the yellow arrows from bottom to top.We want to be sure that inputs, lead to outputs, which lead to outcomes and finally the desired impact.Starting at the bottom are the resources needed. One input is human resources: IC focal point staff identified in each HIV clinic. They participate in an IC training, including a workshop to finalize a national template for a clinic triage policy.The output is written triage procedures in each clinic. The trained IC focal point staff take the national template, adapt it to their settings, and put in place a written triage policy specific to their clinic.The short term outcome should be that coughing patients are identified, separated, and rapidly served.Over the long run, the impact will be less TB spread in the clinics. Fewer health care workers will have TB.Citations:WHO. A guide to monitoring and evaluation for collaborative TB/HIV activities WHO/HTM/TB/ , WHO/HTM/HIV/09.01Global Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit, 2009 version.Diagram adapted from: Dominique MathiotInputInfection Control focal point staff identified and trained
11 Indicators Provide useful tools for supervision at all levels Are we doing the right things?Are we doing them right?Help managers and supervisors determine:What’s going well and should be reinforced and replicated elsewhere?What needs improvement and how can the gaps be addressed?[Review slide]CitationGlobal Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit, 2009 version.
12 Choose indicators that: Are able to measure performanceProvide strategic information needed to make good decisions for managing and improving programme performanceAre reliable and measurable on a regular basisAre clear, with well defined numerators and denominators using standard definitions[Review slide]The next few slides present examples of indicators.You can chose those that are the most useful for supervision of the IC programme at each level.CitationGlobal Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit, 2009 version.WHO. A guide to monitoring and evaluation for collaborative TB/HIV activities WHO/HTM/TB/ , WHO/HTM/HIV/09.01
13 Standardized IC indicators for collaborative TB/HIV activities Proportion of health-care facilities providing services for people living with HIV that have infection control practices that include TB controlProportion of health-care workers, employed in facilities providing care for people living with HIV, who developed TB during the reporting periodWHO recently developed standardized, internationally accepted indicators for monitoring TB/HIV collaborative activities. Here are the two that address IC.While specific to facilities providing care for people living with HIV, they can be applied to any facility with a risk of TB transmission.The first is the proportion of health facilities that have IC practices.The second is the proportion of health-care workers who developed TB.The citation in the Instructor’s notes [see below] provides definitions for these 2 indicators, suggests how to collect the information, and assesses the strengths and limits of these indicators.CitationWHO. A guide to monitoring and evaluation for collaborative TB/HIV activities WHO/HTM/TB/
14 Global IC indicators and targets By 2012, 50% of countries should haveDeveloped a national TB infection control planSet up national surveillance of TB disease among health workersAssessed major health-care facilities and congregate settings for TB ICReported on the implementation of TB infection controlBy 2013, all countriesAs we discussed in the introduction to the class, here are global IC indicators in the 2009 WHO Policy on Infection Control.The targets are for 50% of countries to achieve them by 2012, and all countries by 2013.[Review slide]CitationWHO. WHO policy on TB infection control in health-care facilities, congregate settings, and households WHO/HTM/TB/
15 Examples of national IC indicators How many health care workers had TB in the past year?Have health care workers been trained in the past year?Of the country’s tertiary (referral) hospitals, how many have:A person in charge of TB infection control?A TB IC assessment done in the past year?Training on TB IC conducted in the past year?Here are examples of infection control indicators for the national level.These are part of WHO’s TB data collection form countries completed in 2009.[Review slide]
16 Examples of facility level IC indicators Facility level plan in placePerson responsible for TB ICHealth facility assessment doneTraining conductedTriage and separation of TB patients# TB cases or suspects missed at intakeAdequate ventilation and/or UVGIUse of respirators by HCWs in high risk settings[Review slide]“Missed at intake” means that TB symptoms or history were detected only after the patient enters a private room with a clinician, or after numerous visits while symptomatic, instead of upon first entry into the facility.CitationWHO. TB infection-control in the era of expanding HIV care and treatment. Addendum to WHO Guidelines for the prevention of TB in health are facilities in resource-limited settings
17 Facility-level indicators ImpactNumber of health care workers with TBOutcomeNumber of TB cases or suspects missed at intake to the facilityOutputFacility-specific, written triage procedure existsLet’s go back to our example.Here are some indicators that could be used to during a supervisory visit to a district clinic to monitor the implementation of this part of the country’s IC plan.[Review slide][Ask participants]Which of these indicators may also be useful for a national manager to assess the country-wide implementation of this part of the national IC plan?[Possible responses]Number of facilities with IC focal personsNumber of facilities with written triage procedures to identify and separate coughing patientsNumber of health care workers with TBCitations:WHO. A guide to monitoring and evaluation for collaborative TB/HIV activities WHO/HTM/TB/ , WHO/HTM/HIV/09.01Global Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit, 2009 version.Diagram adapted from: Dominique MathiotInputInfection control focal person is identified and trained
18 Setting targets Gap analysis: baseline compared to need not yet met Constraints to meeting the needFeasibility for scaling upSetting ambitious yet realistic targets is an important element of the planning process.Good targets are based on a comprehensive and up to date analysis of the situation.A gap analysis compares the baseline to the need.Setting targets also requires consideration of constraints to meeting the need, and feasibility for scale up.CitationGlobal Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit, 2009 version.
19 Example of setting a target By 2015, __ of the country’s HIV clinics will have a written policy for identification and separation of coughing patientsGap analysis: baseline is 0, need is 100%Constraints: no template policy, limited training capacity, few IC focal persons in the HIV clinicsFeasibility: resources available to address the constraints and fill the gapLet’s work through an example.You have chosen this outcome indicator (in bold), and need to set the target (to fill in the blank).When you conduct your gap analysis, you find that none of the HIV clinics have a policy, but all need them.You face significant constraints in getting there: there is no sample policy in your country the clinics can use, there are few IC focal persons identified in the HIV clinics, and you have limited capacity to train them on adopting the policy.To set the target, you think through how well you can address these constraints by 2015, and thus, how much of a scale up is feasible.You’ll judge for your own country what is the appropriate target.CitationGlobal Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit, 2009 version.
20 SummaryMonitoring is the routine tracking and reporting of priority information about a programmeUse global indicators to allow measurement of impact across countriesChose additional indicators that provide strategic information needed to make good decisions for managing and improving programme performanceAdapt standardized IC indicators to your country context, and use them for supervision at each level[Review slide]