1. 5th edition NTP MANUAL OF PROCEDURES Monitoring, Supervision and Evaluation

2. Contents Objectives Definition of Terms Policies Procedures
Monitoring and Supervision
Standard Monitoring and Supervision Form
Evaluation
Program Indicators

3. Objectives
Ensure adherence of program implementers to NTP policies and guidelines
Monitor progress of program implementation, identify gaps and provide basis for decision making to improve implementation
Ensure that the program targets of 90% case detection rate (all forms) and 90% treatment success rate (all forms) are reached and maintained
These are the objectives of monitoring, supervision, and evaluation
Emphasize:
Adherence to protocol
Identify gaps in implementation and address gaps
Ensure attainment of program objectives

4. Definition of terms
Monitoring  regular, systematic and purposeful observation of program performance to determine whether activities are implemented as planned and according to schedule
It also involves giving feedback to implementers, program managers, donors and beneficiaries of the program.

5. Definition of terms
Supervision  the process of overseeing the performance and progress of a person or group
It aims to increase efficiency of health workers by developing their knowledge and skills, improving work attitude, and increasing their motivation.
Monitoring is observation of activities (whether being conducted) while supervision is observation of persons or groups.
Emphasize that supervision should motivate health workers.

6. Definition of terms
Evaluation  the careful collection of information about program, or some of its aspects, with particular focus on its effectiveness and impact over time
Evaluation – long term. “Higher” level indicators measured (impact).

7. Policies
The CHD NTP coordinators shall serve as technical assistance providers for the PHO/CHO NTP coordinators.
The provincial or city NTP coordinators shall serve as NTP supervisors for all DOTS facilities.
The DOTS facility physicians shall serve as NTP supervisors for the health staff of the facility
The Public Health Nurse serves as supervisors for midwives. Midwives shall supervise community volunteers.
This policy clarifies the roles of different levels in MSE.
Since LGUs are not administratively under the DOH, CHDs serve primarily as technical assistance providers rather than supervisors.
This slide also demonstrates that DOTS facility staff (physicians and nurses) are also managers/supervisors and not just clinical staff or implementers.

8. Policies
Monitoring, supervision and evaluation activities should be integrated in the annual workplans of the health facility.
Conduct of monitoring and supervisory visits should be done on a quarterly basis.
Areas may be prioritized for monitoring based on TB program performance and other needs.
Whenever feasible, NTP monitoring in DOTS facilities shall be integrated with monitoring of other health programs.
A monitoring plan should be included in annual workplans of DOTS facilities/LGUs. This monitoring plan should contain the list of areas/facilities to be visited, objectives of the visit, timelines, expected outputs and feedback mechanisms
Quarterly monitoring is strongly recommended.
Since it may not be feasible to visit all facilities quarterly, prioritization should be done based on needs.

9. Policies
Qualitative and quantitative data from routine NTP reports shall be analyzed and used to identify and address problems in program implementation.
Key program indicators shalls be used to monitor and evaluate TB program performance at all levels.
Local Government Units shall support monitoring, supervision and evaluation activities.
the last section of this presentation will discuss the key program indicators.

10. Procedures (Monitoring and supervision)
Records and reports review
The usual NTP records for review are presumptive TB masterlist, laboratory and TB registers, treatment cards, quarterly reports and stock inventory cards.
Completeness, accuracy and consistency
Verify if the classification of the patient, the category of treatment and the treatment outcome are correct
Check the treatment card if drug intake is complete and if sputum follow-up is done on time.
There are three possible methods of doing monitoring and supervision:
Review of records and reports
Direct observation of health workers doing their tasks, and
Interview with health workers and patients
This slide shows guidelines in doing review of records
Check data quality---completeness, accuracy and consistency of content between different recording/reporting forms

11. Procedures (Monitoring and supervision)
Direct observation
Routine tasks that can be observed include:
sputum collection
storage of specimens
patient flows
provision of treatment and counselling
storage and inventory of drugs
waste disposal practices and infection control practices.
Observe if the DOTS facility staff are giving correct and relevant information to patients and doing DOT correctly.
The second method is direct observation. This slide shows some of the tasks of health workers that can be observed by the supervisor.
During observation, check for accuracy of messages and information.

12. Procedures (Monitoring and supervision)
Interviews of health workers and patients
to collect information about health workers’ views about program performance, activities and problems. and about patients’ knowledge on TB causation, transmission, and treatment
The 3rd method of doing monitoring and supervision is interview of health workers and patients.

13. Procedures (Monitoring and supervision)
Practice supportive supervision by being tactful, diplomatic and facilitative.
Provide feedback to the facility and personnel monitored and supervised.
Prepare a formal report of the monitoring and supervision conducted. Give a copy of the report to the DOTS facility or personnel visited and to the next higher level.
Remember these points when doing a monitoring visit.
Lastly, always inform the facility you are going to visit for them to prepare the necessary data.

14. Procedures (Standard monitoring forms)
Program Indicators
Records and Reports (Data Quality Assessment)
Laboratory Monitoring
Health Facility And Logistics (Drugs and Supplies)
Health Staff Interview
Patient Interview
Supervisory Checklist
Refer them to the handout for a copy of the standard monitoring form.
This form has seven sections, shown here.
Emphasize that depending on the monitoring plan, the specific objective of the monitoring may differ between visits. So, it is possible to use only several sections of the tool at any given monitoring visit.
Run through the assessment tool quickly. Note that indicators will be discussed at length towards the end of the presentation.

15. Procedures (Evaluation)
All NTP Quarterly reports will be reviewed and analyzed by the DOTS facility physician prior to submission to the PHO/CHO.
Together with the DOTS facility staff
Quarterly reports will be collected, consolidated and analyzed by the PHO/CHO, CHD and NTP. At all levels, identified problems and recommended actions will be given as feedback.
Evaluation can be done periodically and the quarterly reporting is a good opportunity to evaluate past quarter’s performance.
Emphasize: the physician should analyze the data with the DOTS facility staff before submission to next higher level.
At each level, PHO/CHO, CHD, NTP, analysis of consolidated data should be done and findings discussed with the submitting units.

16. Procedures (Evaluation)
LEVEL
FUNCTIONS
TIMELINE FOR SUBMISSION
DOTS Facility
Data collection, analysis and submission to next higher level
1st week of the month following end of quarter
PHO/CHO
Data collection, analysis, consolidation, feedback and submission
2nd week of the month following end of quarter
CHD
3rd week of the month following end of quarter
DOH-NTP
This table shows the tasks of each level– that of consolidating, analyzing, submitting and giving feedback.
This also shows the timeline of report submission, essentially the same as previous MOP.

17. Procedures (Evaluation)
Program implementation review (PIR) will be conducted on a semi-annual or annual basis to evaluate performance.
Plan for the PIR: what program elements to evaluate? what data collection tools and methods to use?
Collect the necessary data prior to the actual PIR.
Analyze and interpret the TB data collected. Report and explain the results of the analysis.
Discuss strategic directions that will address program implementation gaps.
Prepare the PIR report and disseminate.
Analyze the core program indicators (CDR and CR) from the LGU scorecard.
Aside from analysis of quarterly reports, program implementation reviews done annually or semi-annual is another method of evaluating the program.
To have good quality results, it is important to plan the PIR very well. Emphasize that the PIR is not just the actual meeting or workshop but success relies heavily on the preparatory activities, especially data collection and analysis.
Lastly, the LGU scorecard should be analyzed as it is the official LGU indicator on core TB indicators.

18. Program indicators 1. Case Notification Rate (all forms)
2. TB Case Detection Rate (all forms)
3. Notification Rate for MDR-TB
4. Treatment Success Rate (all forms)
5. Cure Rate (new bacteriologically confirmed)
6. Treatment Success Rate for MDR-TB
7. Percent of MDR-TB cases still under treatment after 6 months
8. Total number of presumptive TB examined
9. Percent contribution from non-NTP care providers
10. Number of children with TB detected and given treatment and those given IPT
11. Percentage of TB cases in Category A and B areas with HIV counseling and
testing among aged 15 years old and above
12. Percentage of MDR-TB cases provided with HIV counseling and testing
13. Percent of TMLs within EQA standards
14. Percent of DOTS/laboratory facilities with no stockouts of anti-TB drugs and
laboratory supplies in the last 12 months
Fort the program indicators, there are 14 “core indicators” listed here.
Refer them to the handout on program indicators which lists the definition and formula for each indicator.

19. DEFINITION/ CALCULATION
Program indicators
INDICATOR
DEFINITION/ CALCULATION
DATA SOURCE
1. Case Notification Rate
(all forms)
Number of notified TB cases (all forms) for every 100,000 population
Numerator = No. TB, all forms
Denominator = population divided by 100,000
Note: Notified TB cases refer to “all forms” (include new and relapse—whether PTB or EPTB, and whether bacteriologically confirmed or clinically diagnosed.)
Report 3a. Quarterly report on Case Finding of Drug- susceptible TB Cases and IPT
2. TB Case Detection Rate (all forms)
Percentage of TB cases (all forms) detected and treated out of the estimated incident cases of TB (all forms)
Numerator = Number of TB, all forms detected
Denominator = Total number of all forms estimated to occur each year (i.e., population x incidence rate of TB, all forms) 
Through the exercise, they will compute on their own the program indicators.
Discussion follows wherein the definition of the indicators and calculation are clarified.

20. Report 3a. Quarterly Report on Case Finding of Drug-susceptible TB Cases and IPT
Point out where to get the “all forms” (numerator) to compute the indicator

21. DEFINITION/ CALCULATION
Program indicators
INDICATOR
DEFINITION/ CALCULATION
DATA SOURCE
3. Notification Rate of MDR-TB
Proportion of bacteriologically confirmed RRTB/MDR-TB registered out of the estimated MDR-TB cases among new and retreatment TB cases
Numerator = No. of bacteriologically confirmed drug-resistant TB (RR/MDR-TB) cases registered
Denominator = estimated MDR cases among the new and retreatment TB cases 
 Report 3b. Quarterly Report on DR-TB Cases
4. Treatment Success Rate (all forms)
Percentage of TB, all forms that are successfully treated
Numerator = Number of All forms of TB cases cured and completed treatment
Denominator = Total number of all forms of TB cases registered during a specified period
Report 5a. Quarterly report on Treatment Outcome of Drug Susceptible TB Cases
Through the exercise, they will compute on their own the program indicators.
Discussion follows wherein the definition of the indicators and calculation are clarified.

22. DEFINITION/ CALCULATION
Program indicators
INDICATOR
DEFINITION/ CALCULATION
DATA SOURCE
5. Cure Rate (new bacteriologicallyconfirmed)
Percentage of TB, new bacteriologically confirmed, that are cured
Numerator = No. of new bacteriologically confirmed TB cases cured
Denominator = Total number of new bacteriologically confirmed TB cases registered during a specified period
 Report 5a. Quarterly report on Treatment Outcome of Drug-susceptible TB Cases
6. Treatment Success Rate of MDR-TB
Percentage of MDR-TB that are successfully treated
Numerator = No. of registered bacteriologically confirmed drug-resistant TB cases (RR/MDR-TB) cured and completed treatment   
Denominator = No. of bacteriologically confirmed drug-resistant TB cases (RR/MDR-TB) registered during a specified period   
 Report 5c. Annual Report on the Treatment Outcome of DR-TB Cases
Through the exercise, they will compute on their own the program indicators.
Discussion follows wherein the definition of the indicators and calculation are clarified.

23. Report 5a. Quarterly Report on Treatment Outcome of Drug-susceptible TB Cases
Numerator will be the green circle—”all forms” (new and Relapse) that were cured and treatment completed for both BC and CD.
Denominator will be red circle or TOTAL.
Note: If the total does not match with cases registered the previous year (1st column), this may be due to the number on the footnote or “excluded from cohort because found to be DRTB regimen”.
“Missing cases” (that is=total number of new and relapse registered last year minus total evaluated minus excluded) should be added to the denominator as “not evaluated” so that the discrepancy will be accounted for.

24. DEFINITION/ CALCULATION
Program indicators
INDICATOR
DEFINITION/ CALCULATION
DATA SOURCE
7. Percent of bacteriologically confirmed MDR-TB with negative culture after 6 months of treatment (interim outcome)
Percentage of bacteriologically confirmed MDR-TB cases with negative culture after 6 months
Numerator: No. of bacteriologically confirmed MDR-TB cases with negative culture after 6 months of MDR-TB treatment
Denominator: Total no. of bacteriologically confirmed MDR-TB cases
Report 5b. Quarterly Report on Treatment Interim Outcome of DR-TB Cases
8. Total number of presumptive TB examined
Absolute number of presumptive TB examined during the reporting period
Report 1a. Quarterly Report on TB Microscopy and GX Laboratory Examinations
Through the exercise, they will compute on their own the program indicators.
Discussion follows wherein the definition of the indicators and calculation are clarified.

25. Report 1. Quarterly Report on TB Microscopy and GX Laboratory Examinations
Presumptive examined will refer to those that underwent DSSM.
It does not include presumptive TB children who did not undergo DSSM and does not include Xpert.
Note: reason for this is that under PhilPACT, this indicator was intended as a measure of access to microscopy services.

26. DEFINITION/ CALCULATION
Program indicators
INDICATOR
DEFINITION/ CALCULATION
DATA SOURCE
9. Percent contribution from non-NTP care providers
Percentage of notified TB cases contributed by non-NTP care providers
Numerator = No. of notified TB cases, all forms, referred/managed by all non-NTP care providers (other government, private, community, etc.)
Denominator = Total number of notified TB cases, all forms
Report 3a. Quarterly report on Case Finding of Drug-susceptible TB Cases and IPT
10. Number of children with TB detected and given treatment and those given IPT
Absolute number of children with TB detected and given treatment plus those given IPT
Report 3a. Quarterly report on Case Finding of Drug–susceptible TB Cases and IPT
Through the exercise, they will compute on their own the program indicators.
Discussion follows wherein the definition of the indicators and calculation are clarified.

27. DEFINITION/ CALCULATION
Program indicators
INDICATOR
DEFINITION/ CALCULATION
DATA SOURCE
11. Percentage of TB cases in Category A and B areas with HIV counseling and testing among aged 15 years old and above
Percentage of notified TB cases in Category A and B areas that underwent HIV counseling and testing
Numerator = Number of registered TB cases in Category A and B areas provided with HIV counseling and testing among 15 years old and above
Denominator = Total number of registered TB cases in Category A and B areas 15 years old and above
Report 3a. Quarterly report on Case Finding of Drug- susceptible TB Cases and IPT
12. Percentage of MDR-TB cases provided with HIV counseling and testing among aged 15 years old and above
Percentage of notified MDR-TB cases in Category A and B areas that underwent HIV counseling and testing
Numerator = Number of registered MDR-TB cases provided with HIV counseling and testing among 15 years and above
Denominator = Total number of registered MDR-TB cases 15 years old and above
Report 3b. Quarterly Report on DR-TB Cases
Through the exercise, they will compute on their own the program indicators.
Discussion follows wherein the definition of the indicators and calculation are clarified.

28. Tested for HIV among those eligible
Report 3a. Quarterly Report on Case Finding of Drug Susceptible TB Cases and IPT
IPT among children
Non-NTP Providers
Tested for HIV among those eligible
TB in Children
(add to IPT below)

29. DEFINITION/ CALCULATION
Program indicators
INDICATOR
DEFINITION/ CALCULATION
DATA SOURCE
13. Percent of TMLs within EQA standards
Percentage of TMLs with less than 5% major errors
Numerator = Number of TMLs that have less than 5% major errors (adequate performance)
Denominator = All TMLs providing TB laboratory services within the NTP laboratory network
Report 2. Quarterly Report on EQA for TB Microscopy
14. Percent of DOTS/ laboratory facilities with no stockouts of anti-TB drugs and laboratory supplies in the last 12 months
Percentage of DOTS facilities with no stock-outs of anti-TB drugs and laboratory supplies in the past 12 months
Numerator = No. of DOTS and laboratory facilities with no stockouts of 1st line and 2nd line anti-TB drugs and lab supplies in the past 4 quarters
Denominator = total number of DOTS and Lab facilities
Report 4. Quarterly Report on Drug and Supply Inventory and Requirement
Through the exercise, they will compute on their own the program indicators.
Discussion follows wherein the definition of the indicators and calculation are clarified.

30. Report 2. Quarterly Report on EQA for TB Microscopy
Adequate performance = those with less than 5% ME over TOTAL TMLs

31. Report 4. Quarterly Report on Drug and Supply Inventory and Requirement

32. Thank you