1. Monitoring and Evaluation: Tuberculosis Control Programs

2. Learning Objectives
Understand the principles of M&E for effective TB programming.
Construct conceptual and result frameworks.
Select and make proper use of indicators and data for TB M&E.
Be able to develop a monitoring and evaluation plan.

3. Content Outline Problem statement M&E (definitions)
Opportunities, challenges, and strategies for TB control
Conceptual and results frameworks
M & E frameworks

4. Content Outline…cont’d
Targets
M & E indicators
Source of data
M&E tools for TLCP
M & E challenges

5. Problem statement
1/3 of world population (2 billion) infected with M. tuberculosis.
9 million new cases of TB/year.
2 million deaths/year.
Inadequate Control Programmes.
Speaker note
According to WHO, TB infection is currently spreading at the rate of one person per second. It kills more young people and adults than any other infectious disease and is the world’s biggest killer of women. Every year 8-10 million people catch the disease and 2 million die from it. About a third of the world’s population, or around 2 billion people, carry the TB bacteria but most never develop the active disease. Around 10% of people infected with TB actually develop the disease in their lifetimes, but this proportion is changing as HIV severely weakens the human immune system and makes people much more vulnerable.

6. M&E
What is monitoring?
What is evaluation?

7. M & E
Monitoring
is the routine tracking of programs using input, process and outcome data that are collected on a regular basis.
is used to assess whether or not planned activities are carried out according to schedule.
is usually done by insiders.

8. M&E …cont’d Evaluation
Periodic assessment of programme or project against set targets.
Usually done by outsiders.
Types
Process evaluation
Outcome and impact evaluation

9. M&E…cont’d Process evaluation
is used to measure quality and integrity of programme implementation and to assess coverage
it may also measure the extent to which the intended target population uses services
inform midcourse corrections in the programme

10. M&E…cont’d Outcome evaluation
measures the extent to which stated objectives are achieved with respect to the programme’s goals
assesses influence of programme activities by measuring changes in knowledge, attitude, behaviors, skills, community norms, and health-service utilization.

11. M&E…cont’d Impact evaluation
is used to determine how much the observed change in outcomes can be attributed to specific programme efforts.
involves complex data collection and analysis procedures
assist to determine the success of a project for scale-up or replication.

12. Why M &E? M& E assists in day-to-day management of health programmes.
M&E provides information for strategic planning, programme design and implementation.
M&E assists informed decision-making about human and financial resources, especially in resource-limited settings.

13. Good M&E
ensures the most efficient use of resources to generate the data needed for decision-making.
guides data collection and analysis to increase consistency and to enable managers to track trends over time.
serves as a catalyst to coordination.

14. Opportunities for TB Control
Low cost, accurate diagnosis and treatment available for over three decades.
M & E system is in place.
Speaker’s note
Unfortunately, despite the availability of low-cost and accurate diagnosis as well as nearly 100% curative treatment for more than three decades, tuberculosis remains one of the leading infectious causes of death globally, killing nearly two million people a year.
It is sad to note that there was no rapid progress in the field of tuberculosis control over the past two decades. During the last three decades no single anti-TB was discovered. As a result it was only the level of frustration that grew. However, in recent years, there has been renewed interest in tuberculosis.

15. Challenges of TB Control
Global emergency
- Rising incidence of TB.
- HIV pandemic.
- MDR- TB.
Gaps in coverage, case detection and treatment success
Speaker notes
The World Health Organization (WHO) declared tuberculosis (TB) a global emergency in 1993, in response to a steady increase in the incidence of TB, shifting dynamics in TB disease related to the HIV/AIDS epidemic, and the emergence of multidrug- resistant TB (MDR-TB). The increasing burden of TB is due to many factors, including neglect of TB control by governments; poor management of programmes;
In response to this emergency, the WHO developed a strategy known as DOTS (the internationally recommended TB-control strategy). The DOTS strategy is designed to correct weakness in previous models of program management and to strengthen diagnosis and treatment services.
TB control programmes face many new and existing challenges. Traditionally, a lack of political commitment to TB control, which in turn leads to weak support of TB control activities from the health system and society, continues to be an ongoing challenge in many countries. Similarly, weak public sector health services, which desperately need to enhance their capacity to implement, expand, and sustain DOTS- based services without compromising the quality of case detection and treatment, hinder progress in TB control.
Among the newer challenges, the impact of the HIV/AIDS epidemic on TB incidence is daunting. Even in the presence of well-functioning TB control programmes, the incidence of active disease is increasing in settings with a high prevalence of HIV.
Another challenge is the exponential increase in MDR-TB. This challenge requires effective implementation of the DOTS strategy to prevent new MDR-TB cases.
Although some progress has been made, persistent gaps remain in coverage, case detection, and treatment success – three key global indicators that are recommended by the World Health Assembly for measuring national TB control program (NTP) success.
As a result of all these challenges TB remains to be a leading infectious cause of death globally.

16. Control Strategy (DOTS)
Sustained political commitment.
Access to quality-assured TB sputum microscopy.
Standardized short-course chemotherapy.
Uninterrupted supply of quality-assured drugs.
Recording and reporting system enabling outcome assessment.
Speaker’s note
DOTS is the internationally-recommended strategy to ensure cure of tuberculosis. It is based on five key principles that are common to disease-control strategies, relying on early diagnosis and cure of infectious cases to stop spread of tuberculosis.
The package of interventions that eventually became to be known as the DOTS strategy was first formulated in national tuberculosis programmes supported by the IUATLD under the leadership of Dr Karel Styblo. Initially in the United Republic of Tanzania and then later in several other countries of Africa and Latin America.
The system of recording and reporting under the DOTS strategy has three essential components:
- recording all patients with the diagnosis of TB.
- recording their treatment outcome.
- reporting results to the next higher level.
WHO began to promote this strategy in 1991 and in 1994 produced a framework later known as the DOTS strategy.

17. Basic Assumptions for DOTS
Government commitment avails sufficient funds and administrative support.
Microscopic exams detect the most infectious cases and are affordable.
Direct observation ensures adherence.
Uninterrupted drugs ensure cure.
Recording & Reporting help to monitor and evaluate.
Speaker’s note
Government commitment is an essential component of DOTS, and WHO has emphasized advocacy and social mobilization as means of achieving this commitment. Sufficient funds and administrative support to hire staff, purchase essential items (drugs, microscopes, reagents, printed materials, etc), and contract for services are necessary for the programme to operate.
The rationale for diagnosis primarily by microscopy among patients in health facilities is simplicity of the procedure (i.e., it can be done in remote areas with minimum lab facilities and detects the most infectious cases).
Direct observation of treatment in which “ a trained and supervised person observes the patient swallowing the tablets” is fundamental to the DOTS strategy to ensure adherence to treatment.
The requirement of an interrupted supply of tuberculosis drugs is clear. In addition, the quality of drugs should be ensured, particularly if they are provided in fixed dose combinations, which are more susceptible to problems in manufacture.
The last component is the main source of epidemiological data useful to monitor and evaluate the control programme. DOTS records can be easily checked for internal consistency and for consistency between records, and can also be externally verified by reviewing sputum slides, interviewing patients and health workers, and monitoring consumption of drugs and supplies.

18. Levels of intervention for TB Control
Primary – BCG vaccination
- INH prophylaxis
Secondary – early diagnosis and proper treatment
Tertiary – Prevent complications
Speaker’s note
BCG vaccination protects against severe forms of Tuberculosis (TB meningitis and miliary TB) and provides more protection against Leprosy than TB.
INH prophylaxis is administered to persons not yet infected, to prevent infection, because if that should occur, the person’s natural defenses would put little resistance. This is especially indicated for children in contact with a smear positive patient, up to two months after that patient has become bacteriologically negative as a result of treatment (primary chemoprophylaxis).
INH is also indicated in persons already infected, to reinforce natural defenses especially for persons with immunosuppression (secondary chemoprophylaxis). INH was selected due to its low cost, few adverse effects, oral administration and efficacy.
Some cases of TB (TB of the vertebrae benefit from surgical interventions to prevent neurological sequale).
TB prevention and control relies primarily on secondary level of intervention. This implies early diagnosis and proper treatment of cases. At individual level it helps to treat and cure the patient before his/her condition gets worse and at community level it helps to prevent spread of infection to the community.

19. Conceptual Frameworks – TB Programmes
External Factors Resources
Clinical and managerial staff
Drugs
Laboratories
TB infection
Health Systems (DOT)
Availability
Access
Quality
Utilization
TB Morbidity
Prevalence
Incidence
HIV co-infection
MDR-TB
TB mortality
Speaker Notes
This is an example of conceptual framework for national tuberculosis programmes.
Factors include those external to a programme, such as demographic or socio-economic characteristics. It also includes factors related to the health system and to individuals.
You can see that the programme is aiming to affect the health system and knowledge of TB which will in turn affect morbidity, which will ultimately affect mortality due to TB. Other health related factors that can affect morbidity are included in this model.
Program Factors
Political commitment
Donor involvement
National TB programme
TB knowledge
Case detection
Adherence
Stigma
Co-morbidity
HIV Malnutrition
Alcoholism Diabetes

20. M&E framework for TB programme
OUTPUT
Diagnostic & Treatment services
Improved KAP
Reduced Stigma
OUTCOME
Case detection
Case treatment
Case holding
IMPACT
TB infection
TB morbidity
TB mortality
INPUT
Policy environment
Human and financial resources
Infrastructure
PROCESS
NTP Mgt
Training
Drug Mgt
Laboratories
ACS
An M&E framework is a visual conceptualization of how the elements of a programme fit together, that is , which inputs are necessary for the programme’s activities (process), what outputs are expected from the activities, and what short-and long term outcomes will ultimately result from the programme.
A framework can be used as a tool to understand and analyze a programme, which is crucial for developing and implenting sound M&E plans.
Here is a generic example of an M&E framework for a national TB programme. This framework shows the linear relationships between inputs, processes, outputs, outcomes, and impact. It also provides a visual interpretation of the broad context for the NTP, which includes: Political commitment
Health system variables
Socio-economic conditions
Epidemiological context of TB
The purpose of the framework is to expand on it and create a M&E framework that could be used to guide planning, monitoring and evaluating the directly observed therapy as the standard of care for smear positive TB cases.
CONTEXT
Political commitment Health system Socio-economic conditions Epi-context
Availability HIV prevalence
Access Malnutrition
Utilization Alcoholism

21. Results Frameworks - TB programmes
SO1: Increase tuberculosis case detection to 70%
IRl: Increased availability of quality services
IR2: Increased demand for quality services
IRl.1: Services increased
IR2.1: Customer knowledge of TB improved
Speaker Notes
Successful TB control calls for increased case detection, and that in turn calls for increasing demand and supply.
Note that in this more developed example, the IRs and sub-IRs are phrased in such a way to indicate the desired results. For example, IR1 is not simply “quality services” , but “availability of quality services increased.”
Also note that each IR and sub-IR is measurable. That is, indicators can be developed and data collected to calculate those indicators.
IR : Intermediate Result
IRl.2: Practitioners’ skills and knowledge increased
IR2.2: Social support for TB practices increased
IRl.3: Improved programme management

22. Global Targets (by 2005) Case detection
Detect 70 % of all smear positive TB cases
Treatment outcome
85% of detected TB cases are cured
Speaker’s note
The global targets for tuberculosis control are to detect 70% of the estimated new sputum smear-positive cases and to cure 85% of the sputum smear-positive cases detected. These targets were originally adopted by WHO in It became clear that the global targets would not be achieved by 2000 as intended, and the target date was postponed to 2005 by the World Health Assembly in May 2000.
The numerator for the case detection target is the number of new cases of sputum smear-positive tuberculosis registered in one year, and the denominator is the number of new sputum tuberculosis cases estimated to have arisen in the same population over the same period. As the case detection target relies on an estimate for the incidence of tuberculosis, it is difficult to measure in most settings, especially in the context of an epidemic of HIV infection.
The numerator for the cure-rate target is the number of patients in one-year cohort of new cases of smear-positive tuberculosis fulfilling the WHO definition of cure, and the denominator is the number of patients originally registered for treatment in that cohort.
Adoption of the targets is based on two principles – impact and feasibility. First, epidemiological modeling has demonstrated that achieving the targets will result in a significant decline in the tuberculosis epidemic, reducing incidence by about 50% in 8-12 years, in the absence of HIV. Second, the targets took into account accessibility and affordability of services and health-seeking behavior of the population.

23. Indicators Valid Reliable Specific Sensitive Operational Affordable
Feasible
Comparable
Valid: indicators should measure the condition or event they are intended to measure.
Reliable: indicators should produce the same results when used more than once to measure the same condition or event, all things being equal (e.g., using the same methods, tools , or instruments).
Specific: indicators should measure only the condition or event they are intended to measure.
Sensitive: indicators should reflect changes in the state of the condition or event under observation.
Operational: indicators should be measured with definitions that are developed and tested at the program level and with reference standards.
Affordable: the costs of measuring the indicators should be reasonable.
Feasible: it should be possible to carry out the proposed data collection.
Comparable: indicators should be comparable (e.g., over time, across geographical lines).

24. MDGs (by 2015)
Goal 6 : to combat HIV/AIDS, malaria, and other diseases
Target 8: to have halted and begun to reverse the incidence of malaria, TB, and other major diseases by 2015
Indicator 23: between 1990 and 2015, to halve the prevalence and death rates associated with tuberculosis; and
Indicator 24: by 2005, to detect 70% of smear positive and successfully treat 85% of these cases.
Speaker’s notes
The MDG framework consist of a hierarchy of indicators that measure progress towards “targets”, which are the specific achievements needed to satisfy higher “goals.” Those most directly relevant to TB control are Goal 6 (to combat HIV/AIDS, malaria and other diseases) and Target 8 (to have halted by 2015 and begun to reverse the incidence of malaria, TB, and other major diseases). Among the indicators for Target 8 are two groups that can be used to evaluate the implementation and impact of TB control: indicator 23 and 24.
The target for case detection and treatment have been set by WHO’s World Health Assembly (Resolution WHA 44.8).
The targets for prevalence and deaths are based on a resolution of the year 2000 meeting of the Group of Eight (G8) industrialized countries, held in Okinawa, Japan.

25. Global Indicators TB case detection. Treatment success rate.
DOTS coverage.
Surveillance of multi-drug resistant TB.
HIV seroprevalence among TB patients.
Speaker’s note
The five global indicators described in this section are based on data reported by National TB Programmes. Data are used to monitor progress in DOTS expansion and achievement at national and global levels of the WHO targets for TB control: case detection rate of at least 70% and treatment success of at least 85%.
The first three indicators are reported to WHO every year by national TB programmes and are included in the annual WHO report on global TB control. These indicators measure NTP progress towards international targets for case detection, treatment success, and DOTS coverage.
The last two indicators provide important information on whether countries are aware of the prevalence of MDR-TB and HIV among TB cases. It is important to monitor whether or not NTPs are performing surveillance to estimate the prevalence of HIV among TB cases, and vice versa, because these data is needed to make decisions with regard to collaborative programs.
National surveillance is an important tool for monitoring trend in epidemiological indicators. These trends will in turn indicate the success of control programme. These indicators are, critical to monitoring, evaluation, and problem-solving at national and local levels.

26. Programme-outcome indicators
Case-notification rate (all forms of TB)
Case-notification rate (new smear-positive cases)
Re-treatment of TB cases
Smear-conversion rate
Cure rate, Treatment-completion rate
Treatment-failure rate
Default rate
Death rate
Speaker’s note
The indicators listed above are routinely calculated by TB control programmes at district, regional , and national levels, and they are based on data from the TB registers and quarterly reports on TB case registration, smear conversion, and treatment outcomes. They are used to monitor progress towards achievement of the national targets for case detection and treatment outcomes and to monitor programme quality and effectiveness.
Indicators are used as tools to measure whether patient-oriented activities and programme management activities are reaching their target.
Outcome indicators are used to monitor and evaluate performance. They are considered the optimal level of achievement. However, they should not become a fixed ‘target’, which may lead to falsified data or neglect of one function in favour of other priorities.
Due to the difficulty in measuring the epidemiological impact, outcome indicators are often used as a surrogate for indicators of impact.

27. Sources of Information
Record forms at the health facility
Record and report forms at the district level
Laboratory records
Report forms at the regional level
Report forms at the national level
Speaker’s notes
Once a TB programme has designed and adopted an M&E framework and selected the appropriate indicators, data-collection strategies need to be selected. There is a variety of methods that are typically used to gather TB information. No single data source can provide all of the information required for M&E – a combination is necessary.
Routine data collection at TB treatment facilities and microscopy units is the most common way of collecting TB data for patient and treatment facility management, for monitoring resources used and services provided, and for disease surveillance. Data are recorded by the health staff at the facility or microscopy units while they perform their daily health care activities. These data are recorded on standard reporting forms, which are sent to basic management units (BMUs), where they are aggregated and sent to the national level.
The district, regional, and national TB offices are responsible for their respective geographic areas. Monitoring is often required on a monthly or quarterly basis using several different data-collection tools. The forms listed above are the most commonly-used ones.

28. M&E tools for TLCP Supervision checklist
- checklist for programme management
- checklist for health facility
Review meeting
- annual and semi-annual
- central, regional and district
External Quality Assurance

29. Additional sources of Information (Special studies)
Prevalence surveys
Population-based surveys
Health-facility surveys
Vital registration surveys
Tuberculin surveys
Drug-resistance surveys
Speaker’s note
Special surveys or studies may be needed to determine many of the epidemiological and behavioral indicators that are not collected through the routine recording and reporting. Such studies are often more comprehensive than standard collection, but at the same time, they are more costly and require specific technical capacity for implementation. These factors limit the number of special studies that are conducted.

30. M & E challenges in TB Incomplete recording and reporting
Inconsistent data collection
Lack of timeliness
Inappropriate use of information
Speaker note
An M&E system is only as good as the data that are collected. The data should be appropriate, complete, consistent, and timely. Many current efforts at data collection, particularly those conducted routinely, result in poor-quality data because of a lack of proper training and supervision. If the individuals recording the data are not using the data and do not fully appreciate data needs for programme management beyond the facility level, the quality will most likely be poor. This in turn leads to declining use. One of the key functions of an M&E system is to oversee all data collection and ensure that data are appropriately used and the results are disseminated throughout the system, but especially to the collection level. Changes in health programs that are directly based on evidence from the field reinforce the efforts at the peripheral level to complete routine reporting. When health workers understand the importance of the data they are collecting, quality is likely to improve, building more confidence in and use of monitoring data.

31. Level of M&E in TB: The “ONION”
estimated TB cases
all true TB cases
cases presenting to health facilities
cases presenting to public health facilities
cases presenting to DOTS facilities
cases correctly diagnosed by DOTS facilities
diagnosed cases reported by DOTS facilities
This “onion”, developed by Chris Dye of the World Health Organization, is one way of illustrating how monitoring and evaluation differ, and why we need monitoring and evaluation to guide our programmes.
With a good M&E system, we will be able to track programme successes for the core of the onion and the first or second layer through monitoring activities by use of existing R & R system. The outermost layers require more than monitoring data – we will need an evaluation with rigorous methodology to determine if our programme is having any effect at this level.
With quality R&R the reported number of cases reflect the number diagnosed. With good quality of service (i.e., diagnostic service) the number of cases presenting to DOTS and those correctly diagnosed is similar. Availability and strength of PPM-DOTS determine the proportion of cases presenting themselves out of all presenting to all health facilities. Accessibility, affordability and health-seeking behavior determine the proportion of cases presenting themselves among the true cases.
Chris Dye, 2002

32. References
Compendium of Indicators For Monitoring And Evaluating National TB Programmes. Stop TB Partnership August 2004.
2. Toman’s Tuberculosis Case Detection, Treatment, And Monitoring. Second Edition WHO Geneva 2004
3. WHO REPORT 2005 GLOBAL TB CONTROL Surveillance, Planning, Financing