MEASURE Evaluation Data Quality Assurance Workshop Session 3 Introduction to Routine Data Quality Assessment

Topics to Cover  RDQA vs DQA tools  Data quality responsibilities by level  Introduction to RDQA Objectives Methodology Core components Timeline Preparation for conducting an RDQA  After the RDQA

DQA and RDQA DQA & RDQA were developed for verifying the quality of reported data and assessing data management & reporting systems Data Quality AuditRoutine Data Quality Assessment FunctionEvaluationMonitoring By WhomExternal audit teamProgramme/project team DistinctionHighly structured, external audit Simplified version of DQA for use by programmes

Data Quality Tools  Routine Data Quality Assessment Tools: The Single Indicator tool -- used to assess 1 indicator only The Multi Indicator tool -- used when 1-4 indicators need to be assessed from the same data flow The Longitudinal tool -- used to assess 1 indicator over 4 reporting periods  The PRISM (Performance of Routine Information System Management) Framework is another tool used for routinely assessing the strength of health information systems. One major difference is that PRISM includes a behavioral assessment component.

Objectives of RDQA Verify  Rapid verification of quality of reported data  Capacity of information systems vs. data quality Implement measures to:  Strengthen data management, reporting systems  Improve data quality Monitor  Performance of data management, reporting systems  Capacity to produce quality data

RDQA Methodology  Data Verification: documentation reviews Accuracy  re-counted vs. reported results Reliability  cross checks  Reporting Performance Timeliness, completeness of reporting, availability of reports  System Assessment Are elements in place to ensure quality reporting?

Data Quality Responsibilities by Level M&E Unit Intermediate Aggregated Site Service Delivery Point Data Quality Responsibilities M&E Unit Provide lower reporting levels with clear guidelines on data collection and reporting Disseminate national policies related to data quality Conduct routine supervisory visits to lower levels Provide organogram of positions/data management responsibilities For RDQAs: Initiate RDQAs in conjunction with other national program units Follow up on late, incomplete, inaccurate, or missing reports Capture all data quality checks not yet captured in an electronic format, including spot/cross check, validations, and updates to error logs Intermediate Aggregate Site Follow appropriate procedures to compile service delivery site forms monthly and send report to the National M&E Ensure budget includes funds for data quality activities Follow up on late, incomplete or missing information Conduct routine supervisory visits to service delivery sites For RDQAs: Initiate RDQAs for service delivery sites Follow up data verification checks as part of supervisory visits Document how discrepancies have been resolved Service Delivery Point Summarize patient data & check data quality of patient registers Submit monthly summary reports to the Health District Routinely analyze & use data to improve quality of care For RDQAs Health Sites do not initiate RDQAs

Steps in Conducting an Assessment  Verify and validate performance information to ensure that data are of reasonable quality  Review data collection and processing procedures to ensure consistent application  Review program capacity and human resources  When data quality issues are identified, take steps to address them develop and implement a budgeted action plan for strengthening the system

Core Components of RDQA Data verifications Quantitative Compares recounted to reported data System Assessment Qualitative Assesses strengths and weaknesses of functional areas of M&E system

Data Verification  Observe or Describe  Connection between the delivery of services/commodities and the completion of the source document that records that service delivery  Review Source Documents  Review availability and completeness of all indicator source documents for the selected reporting period  Verify reported data  Cross-checks  Perform “cross-checks” of the verified report totals with other data- sources  Spot checks  verify the actual delivery of services or commodities to the target populations

System Assessment  M&E Capabilities, Roles and Responsibilities  Training  Data Reporting Requirements  Indicator Definitions  Data-collection and Reporting Forms and Tools  Data Management Processes  Data Quality Mechanisms and Controls

When to Assess  Integrate data quality control mechanisms into standard operating procedures  Integrate data quality checks into routine supervisory visits  Conduct periodic formal assessments Full RDQA vs. Data Verification only  Timeline can be different but this is what we suggest

 Determine purpose for conducting an RDQA  Determine levels and sites  Identify indicators, data sources, reporting periods  Prepare for site visit Notify site visits two weeks in advance

After the RDQA  Review the output of the RDQA  Develop a system strengthening plan, including follow-up actions  Plan on sharing the outcome with the levels and sites that participated in the RDQA

Questions & Answers

MEASURE Evaluation is funded by the U.S. Agency for International Development (USAID) and implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill in partnership with Futures Group, ICF International, John Snow, Inc., Management Sciences for Health, and Tulane University. Views expressed in this presentation do not necessarily reflect the views of USAID or the U.S. government. MEASURE Evaluation is the USAID Global Health Bureau's primary vehicle for supporting improvements in monitoring and evaluation in population, health and nutrition worldwide.