Indicators, Data Sources, and Data Quality for TB M&E
Published byModified over 6 years ago
Presentation on theme: "Indicators, Data Sources, and Data Quality for TB M&E"— Presentation transcript:
1 Indicators, Data Sources, and Data Quality for TB M&E This presentation discusses some of the most important factors influencing which indicators we use.Important to review this topic before doing the indicator selection activity.Data sources differ in every context, country, and region so as I go through this presentation, please try to think of data sources and data quality issues that your country uses and has trouble with, so that we can discuss them after the presentation.
2 Criteria of good indicators Valid: indicator actually measures the phenomenon it is intended to measureReliable: indicator produces the same results when used repeatedly, so it always measures the same phenomenonSpecific: indicator measures only the phenomenon it is intended to measureSensitive: indicator reflects changes in the phenomenon under studyOperational: indicator is measured with developed and tested definitions and reference standardsThese ideas are probably review and were discussed in “how to use the compendium.”Indicators are signs, clues, and markers as to how close we are to our path and how things are changing.
3 Qualitative vs. Quantitative Qualitative: indicators answer questions about how well the program elements are being carried outQuantitative: indicators measure how much and how manyExamples of qualitative M&E tools:Sign-in logs,Registrations, forms, registers, checklistsProgram activity formsPatient chartsStructured questionnairesQualitative M&E tools:Focus groupDirect observation
4 Factors in Indicator Selection What national, district and local levels need to knowAvailability of the dataAvailability of human and financial resources to manage the dataProgram needsLender requirements
5 TB data collection methods and sources Routinely collected dataProcess monitoring and evaluationProgram evaluation/reviewsGlobal TB reportingSpecial surveysOnce you have designed a framework and picked appropriate indicators, data collection strategy needs to be developed.Of course, you should have been thinking of data availability while developing the framework and indicators but now you can focus on collection and analysisNo single source can satisfy data needs for M&E.
6 Routine Recording TB Register TB Treatment Card Laboratory Register Cough registerThese are the basic units of data collected at TB treatment facilities and microscopy units. To ensure data quality, it is important that:These cards are filled out correctlyThese units are filled out in the same wayWhat this often requires is:An easy-to-use card (pre-tested)TrainingFrequent review of how nurses, lab techs are filling them out (supervision and feedback)Really communicating why it is important to have this information filled out correctly and timely
7 Routine Reporting District TB Register Quarterly report of new cases and relapses of TBQuarterly report on results of treatment of pulmonary TB patients registered months earlierMonthly or quarterly reporting forms are sent to Basic Management Unit (BMU), where they are aggregated and then sent to the higher level.Assuming the basic units are filed out correctly, we want to ensure that the “transfer” to the aggregated level is done correctly and in the same fashion.Another issue is timeliness.Again, requires supervision and feedback.
8 Process Monitoring and Evaluation Analysis of recording and reportingSupervisionRecords of trainings held, meetings held, events, etc…
9 Program Evaluation/Review Comprehensive review of the entire programConducted every 2-5 yearsExternal and internal experts break up into groups and cover a representative sample of the countryUsually provides input for developing or revising the medium-term development plan
10 Global Reporting: estimated incidence, case notification, treatment outcomes, some budget information and coverage.Can use the numbers from WHO to track your own country over time.Source: WHO (2004). Global Tuberculosis Control: Surveillance, Planning, Financing.
11 Special studies Prevalence survey Population-based survey Facility surveysVital-registration surveysTuberculin surveysDrug-resistance surveysRead from blurbs on special studies in the intro part of the compendium.
12 Example of a national level data-collection system Prevalence SurveyPrevalence SurveyDRSDRSFacilitysurveyFacilitysurveyFacilitySurveyDRS – Drug Resistance SurveyIf appropriate, all of these activities should be budgeted for in a GFATM proposalA facility survey could be considered a TB-Program ReviewExternal Monitoring VisitsRoutine information system & surveillance2000200220042006
13 Why is data quality important? The primary function of health-information systems is to provide data that enhance decision-making in the provision of health services.By ensuring high-quality data, the health information system attempts to guarantee that decision-makers have access to unbiased and complete informationYour data source is only as good as the data it produces. While ensuring that the importance of data quality has been addressed in the previous slides, it is important to spend a few moments thinking about this issue of good data specifically.It isn’t just for feeding information into an indicator that needs to be reported elsewhere, it is important to see these as tools for decision makers at national, district, and local levels to improve their programs.
14 Standards for good quality data Validity: Do the data clearly and directly measure what was intended to measure?Integrity: Are mechanisms in place to reduce the chance that data are intentionally manipulated?Precision: Are the data at the appropriate level of detail?Reliability: Would you come to the same finding if the data collection and analysis were repeated?Timeliness: Are data available frequently enough to inform decisions?
15 Impediments to good data quality Inappropriate data-collection instruments and proceduresPoor reporting and recordingErrors in processing data (editing, coding, data entry, tabulating)
16 What can be done to improve and ensure data quality? Keep the design of the information system as simple as possibleInvolve users in the design of the systemStandardize procedures and definitionsPre-test data-collection instruments to make sure they are useful and user-friendlyEnsure that data collected are useful to the data collectorRegular supervision and feedback from supervisorsPlan for effective checking procedures (such as cross-checking)Training (data collection instruments, data processing, analysis and decision-making based on evidence)
17 Data-quality assessments Example at district level:Step 1: Interview appropriate individual to obtain understanding of data collection, analysis, and maintenance processStep 2: Review reports to determine whether they are consistentNow that we have discussed basic kinds of data sources and the importance of ensuring data quality, let’s walk through addressing data quality in a step-wise fashion.
18 Data-quality assessments (con’t) Step 3: Periodically sample and review data for completeness, accuracy, and consistencyIndicator definitions are consistent with NTP guidelinesData collection is consistent from year to yearData are complete in coverageFormula used to calculate indicator (if any) is applied correctlyStep 4: Compare central-office records with district or district with facility for consistency and accuracy
19 Data-quality assessments (con’t) Possible data-quality limitations:Validity: The reported data do not accurately represent the population. For example, records may over-report or underreport certain parts of the populationIntegrity: The data could be manipulated for a variety of reasonsTimeliness: If reporting is not up to date, than decisions may not based on most recent evidenceReliability: Implementation of data collection may be irregular or mistimed
20 Conclusion needed based on context/region and what the biggest issue is