1. Monitoring and Evaluation Module 12 – March 2010

2. Project Partners Funded by the Health Resources and Services Administration (HRSA)

3. Module Overview  Monitoring & Evaluation (M&E) framework and components  Records, registers and reports  Recording and evaluating response to TB treatment regimens  Supervision International Standards 13 and 21

4. Learning Objectives At the end of this presentation, participants will be able to:  Describe what is meant by “Monitoring and Evaluation”  Discuss the importance of collecting data and ensuring the accuracy of the data  Explain ways in which the data are used to evaluate treatment  Describe how M&E activities can benefit both TB and HIV/AIDS programs

5. Monitoring & Evaluation System  A key element of the Stop TB Strategy  Allows programs to: Monitor progress and treatment outcomes of individual patients Evaluate the overall performance of the TB program at various levels (local, district, national) Identify areas of program improvement and weakness Ensure accountability

6. Monitoring  What is it? Routine tracking of services and program performance

7. Monitoring (2)  How is it done? Through information collection, data input, analyzing the data, and reporting what is found in that analysis outcome report  Why should we do it? To better assess how well a policy or program is achieving its intended target

8. Evaluation  What is it? Episodic assessment of results that can be attributed to program activities Types of evaluation related to M & E: ➜ Process evaluation: assesses the progress in program implementation and coverage ➜ Outcome and impact evaluation: measures the effect of the program activity on the target population

9. What are the Targets?  Stop TB Partnership/WHO 70% TB case detection and 85% cure rate by 2005  Millennium Development Goals (MDG): Halt, and begin to reverse, the incidence of major diseases such as HIV/AIDS and TB by 2015  Decrease TB prevalence and death rates to 50% of the 2000 estimates  United Nations General Assembly Special Session (UNGASS) – global targets

10. Where do Indicators Fit In?  Indicator: a specific, observable, and measurable characteristic or change that shows the progress a program is making toward achieving a specific outcome  Indicators may be expressed in terms of: Number Rate Proportion Percentage

11. Limitations of Indicators Indicators DO NOT:  Measure everything  Tell us why a problem may exist or how to fix it  Determine if problems identified are amenable to intervention  Tell us which interventions are most cost effective

12. What are some possible uses of data collected by the National HIV/AIDS and TB Programs?

13. Using and Disseminating Data M & E can improve and enhance NAP and NTP work by:  Identifying areas of strengths and weaknesses  Helping plot progress toward program goals  Allowing a program to see trends and to identify high risk groups in order to better target TB control efforts  Providing justification for needed resources  Identifying training and supervision needs  Increasing public awareness about TB  Advocating for policy changes and allocation of funds

14. Monitoring & Evaluation Framework

15. Types of M & E Activities  Staff training  Supervision  Completion of reporting forms  Discussions during staff meetings  Ensuring medicine and laboratory stock supplies  Quality control activities  Analysing data and preparing reports

16. Standard 13: Recording & Reporting A written record of all medications given, bacteriologic response, and adverse reactions should be maintained for all patients

17. Standard 21: Recording & Reporting All providers must report both new and re- treatment tuberculosis cases and their treatment outcomes to local public health authorities, in conformance with applicable legal requirements and policies

20. Why is accurate reporting and record keeping important?

21. Data Quality Assurance  Ensures that the information collected adequately represents the program’s activities  Accurate data – measuring what it is intended to measure  Reliable data – collected and measured the same way by all program personnel over time

22. Reporting Forms and Registers  Request for Sputum Examination  Tuberculosis Treatment Card  Tuberculosis Identity Card  Basic Management Unit TB Register  TB Laboratory Register  Quarterly Report on TB Case Registration  Quarterly Report on Sputum Conversion  Register of TB Suspects  Quarterly Report on Treatment Outcomes  Yearly Report on Program Management in Basic Management Unit

23. Tuberculosis Treatment Card

24. Tuberculosis Treatment Card (2) (RH) (RHE)(Other)

25. Preparing a TB Treatment Card Activity

26. Tuberculosis Identity Card  This card contains an extract of information on the treatment card  It is given to the patient at the start of treatment  It is used to record daily DOT and must be used during the intensive phase of treatment  It also serves as a reference document for TB status after treatment  It should be presented to the doctor whenever the patient falls ill in the future

27. Tuberculosis Identity Card (2)

28. Basic Management Unit TB Register  This revised register is the cornerstone of an NTPs monitoring & evaluation system  It records essential information for notification & treatment outcome by district  It should always be kept up to date with data on sputum smear examinations and treatment outcome  Where electronic data collection systems are available, the information from the register should be entered into the database at least once every month

31. Treatment Outcomes  Cured  Treatment completed  Treatment failure  Died  Default  Transfer out

32. Treatment Outcomes CurePatient whose sputum smear or culture was positive at beginning of treatment but who was smear- or culture-negative in the last month of treatment and on at least one previous occasion Treatment Completed Patient who has completed treatment but who does not meet the criteria to be classified as a cure or a failure Treatment Failure Patient who is sputum smear-positive at five months or later during treatment – or – Patient found to harbor a MDR strain at any point of time during treatment, whether smear- negative or -positive

33. Treatment Outcomes (2) DiedPatient who dies for any reason during the course of treatment DefaultPatient whose treatment was interrupted for 1 month or more Transfer Out Patient who has been transferred to another recording and reporting unit and whose treatment outcome is not known. Treatment Success A sum of cured and completed treatment (smear-positive or culture-positive patients only)

34. Supervision

35. How is supervision used in your TB and HIV/AIDS Prevention and Control Programs?

36. Role of Supervision in M & E  Supervision is a process of guiding, helping, training, and enabling staff to improve their performance in order to provide high quality health care services

37. Purpose of Supervision  Provide leadership and direction to staff  Ensure effective program implementation  Monitor operations and evaluate achievement of goals  Ensure adherence to laws and policies  Avoid confusion or duplication of efforts

38. Purpose of Supervision (2)  Monitor that all necessary tasks are properly performed  Ensure that resources are properly used and are available to staff, including training and supplies to carry out their duties  Ensure accountability

39. Barriers to Effective Supervision  Lack of commitment  Lack of proper planning and time management  Lack of tools for Monitoring & Evaluation  Insufficient staff  Problems with transportation  Lack of confidence or preparation

40. Roles and Responsibilities  National Level – NTP and NAP Planning, implementing, monitoring, and evaluating Program at all levels  Regional/District/Parish Levels Coordinating, supervising, planning, implementing, monitoring and evaluating all aspects of TB and HIV/AIDS Programs in the region

41. How Can M&E Information be used in TB and HIV Programs?  Identify gaps in performance  Monitor treatment outcomes  Measure the impact of an intervention or policy change  Identify populations for enhanced control and prevention efforts  Identify local problems as they arise  Ensure high quality TB and HIV prevention and control strategies are consistently provided

42. Summary: ISTC Standards Covered Standard 13: A written record of all medications given, bacteriologic response, and adverse reactions should be maintained for all patients Standard 21: All providers must report both new and re-treatment tuberculosis cases and their treatment outcomes to local public health authorities, in conformance with applicable legal requirements and policies

43. Summary  Several approaches are used to monitor and evaluate TB and HIV/AIDS programs including supervision, training and the keeping of records and registers  Accurate and timely reporting and record keeping is important. It allows true assessment of Program achievements