1. Compendium of Indicators for Monitoring and Evaluating National Tuberculosis Programs

2. Objectives of Session
Provide an overview of the development of Compendium
Explain the organization of the Compendium and how indicators are used
Provide examples of how each subheading for an indicator guides selection/use of indicators

3. What is the Compendium?
A comprehensive and standardized collection of the most widely used and recommended indicators for monitoring and evaluation of National TB Programs.

4. Who is it for?
NTP managers, data managers, regional and district officers
NGO program managers/data managers involved in TB programming
Evaluation specialists
Health system planners (HMIS etc)
Anyone with responsibility for collecting, processing, analyzing, and presenting data on tuberculosis programs.

5. Specific Objectives
Standardize M&E terminology across indicators and programs
Encourage consistent use of indicators to monitor and evaluate programs
Provide guidance for the development of comprehensive evaluation plans
Serve as a resource for the different components of the monitoring and evaluation process
Bullet 1 – This refers to input, process, output, outcome, impact. The idea is to get everyone on the same “page” in terms of how we think about M&E
Bullet 2 – Also aimed at standardization, but in terms of what is actually reported. Making sure everyone uses the same numerators and denominators, criteria for assessing qualitative indicators.
Bullet 3 - Provide guidance for the development of comprehensive evaluation plans, including selection of indicators to measure progress in specific areas
Bullet 4 – Go beyond indicators, provide information/guidance on planning for M&E and strengthening M&E systems.

6. Current status of TB M&E
Patient follow up/case management using WHO standardized forms
Small number of indicators focusing on outcomes of DOTS implementation
Project-specific monitoring forms
Periodic assessment visits at facility level
Focus is on disease control, where a patient centered or quality approach may serve us better.

7. Why a new TB M&E Guide? (1) New Global Initiatives
Global Fund for AIDS, TB & Malaria
STOP TB Partnership
Increased USAID involvement
TB/HIV initiatives
DOTS Plus
Global Fund for AIDS, TB and Malaria, which recommends 10% of program budgets go to M&E
Stop TB – encouraging M&E under expanded DOTS framework
USAID – went from spending very little in 1998 (8 million) to over 200 million last year.
Links between TB and HIV epidemic
DOTS Plus – to qualify for drugs from Green Light Committee, countries needed more systematic monitoring and evaluation

8. Why a new TB M&E Guide? (2) Need for a broader view of M&E
Inputs-processes-outputs-impact: allows better understanding of how to achieve impact
Standardized guidance for global use
Program-based to complement case-management
Program-specific indicators for different settings, types of programs

9. M&E Framework for TB programs
INPUT
Policy environment
Human and Financial Resources
Infrastructure
PROCESS
Management
Training
Drug management
Laboratories
Communication
Advocacy
OUTPUT
Diagnostic services
Treatment services
Improved knowledge, attitudes, and practices
Reduced stigma
OUTCOME
Case detection
Treatment success
IMPACT
Prevalence of TB infection
Prevalence of TB disease
TB morbidity
TB mortality
Red – this is the core of M&E prior to Compendium development
It provides a visual of how all the elements fit together
Tool for understanding and analyzing a program
Process and output are murky areas
p7, Compendium of Indicators for Monitoring and Evaluating National Tuberculosis Programs
USAID, MEASURE, CDC, WHO, IUATLD, KNCV, MSH. WHO/HTM/TB/ , August 2004

10. M&E Framework for TB Programs
INPUT
Policy environment
Human and Financial Resources
Infrastructure
PROCESS
Management
Training
Drug management
Laboratories
Communication
Advocacy
OUTPUT
Diagnostic services
Treatment services
Improved knowledge, attitudes, and practices
Reduced stigma
OUTCOME
Case detection
Treatment success
IMPACT
Prevalence of TB infection
Prevalence of TB disease
TB morbidity
TB mortality
Red shows how Compendium contributes to expanded M&E activities through use of M&E framework, new indicators for key DOTS elements
It provides a visual of how all the elements fit together
Tool for understanding and analyzing a program
Process and output are murky areas
p7, Compendium of Indicators for Monitoring and Evaluating National Tuberculosis Programs
USAID, MEASURE, CDC, WHO, IUATLD, KNCV, MSH. WHO/HTM/TB/ , August 2004

11. Compendium Development
Step 1: Assessment of existing M&E systems within National TB programs and MOH
Step 2: Create an international TB M&E Working group to develop and review indicators
Step 3: Field test indicators in selected countries
Step 4: Build capacity in M&E to collect, disseminate and use information
Overview of how Compendium was developed.

12. Step 1: Assessment of current M&E systems
Field visits to examine M&E systems: data collection forms, reporting, supervision, data use
South Africa, Russia, Honduras, Philippines
Met with NTPs, USAID missions, WHO, CDC, local implementing partners
Review of literature on TB indicators
Purpose of assessment visits was to see what was already out there – we did not want to increase the burden of data collection but also knew that many programs already collected so much info – wanted to see if we could use data that are collected but not reported or even used to identify problems or for program improvement.
Countries chosen based on region and presence of specific program types and challenges.
In each country – met with key partners and visited faciity, district and national level offices to review data collection and use.

13. Results from assessment visits
Substantial amount of data collected at facility level that is not reported
Weakness in reporting mechanisms for facility level data
Few indicators on political commitment, IEC activities, drug supply and TB/HIV
Lack of data from private sector physicians
Findings – go over these. Third bullet – case detection and treatment outcome data already there.

14. Step 2: Creation of international working group
Similar goals to develop more informative indicators on program implementation
Bring expertise from a wide variety of sources:
Stop TB, WHO, UNION, KNCV, CDC, USAID, World Bank, MSH, MEASURE/Evaluation

15. Results of TB M&E Working Group
Indicators for DOTS: Measure key aspects of the TB epidemic in a country and the programmatic response
Based on WHO recommendations and collected through existing systems
External & Expert review

16. Step 4: Building capacity
Step 3: Field testing
Peru, Kazakhstan, Haiti & Thailand
Revision of indicators based on field testing results
Step 4: Building capacity
Egypt (March ‘05), Mexico (April ‘05), Tanzania (September ‘05), India (January ‘06), Eastern Europe (May ‘06)
Technical assistance

17. Indicators (1) Global indicators (5)
Case detection
Treatment success
DOTS coverage
HIV seroprevalence among TB cases
Surveillance of MDR-TB
Routinely reported program outcomes
Smear conversion
Treatment outcome

18. Indicators (2)
Indicators to measure DOTS implementation under expanded framework:
Political commitment (12)
NTP annual workplan and budget
Diagnosis (7)
Existence of comprehensive laboratory network
Case management, including DOT (2)
Proportion of patients with correct prescription
Drug management (8)
Existence of a quality assurance system for drug management
Each component of expanded DOTS frameword and an example of an indicator from each category.

19. Indicators (3)
Indicators to measure DOTS implementation under expanded framework:
Recording & reporting (2)
Accuracy of reports sent to NTP
Supervision (2)
Existence of supervision guidelines
Human resources development (3)
Proportion of health centers with at least one professional trained in the DOTS strategy
Health systems (1)
Equitable distribution of DOTS