Performance Audit The GAVI Alliance Data Quality Audit experience May 2008 ICGFM conference Dr. Mercy Ahun GAVI Alliance Secretariat, Geneva

Performance Audit DQA history and rationale DQA overview Requirements Report Content GAVI DQA Experience Strengths of DQA Limitations of DQA Future Opportunities

History and rationale for DQA History: DQA emerged as an assessment tool to accompany the investment component of GAVI’s Immunization System Support (ISS) window Rationale: DQA aims to provide GAVI with assurance that the level of reward payable through ISS is based on a data validation process

Overview of Data Quality Audit Goal: assure that management of immunisation services are based on sound and accurate data Operational objectives: 1. assess the quality, accuracy, timeliness and completeness of immunisation reporting systems 2. audit the reported DTP3 coverage in infants in a specific calendar year and then estimate the National verification factor to use in reward allocations

Requirements Countries receiving Immunisation Services Support are required to conduct DQA during 2 nd year of support External audit team engaged to review records & reports 4 districts randomly selected & data for previous year reviewed 6 Health facilities visited per district Number of vaccinations checked at facility, district & national levels

DQA report content Performance indicators (DTP drop out rate, wastage, coverage, Verification factor) Verification factor –Estimation based on all DTP3 records and reports audited at facility & district level –Based on ratio of recounted DTP3 and reported DTP3 (consistency ) –Successful DQAs – VF of at least 0.80 Quality of the system index: Composite index based on the recording, reporting, storage, M&E and system design for data

DQA reporting process In-country presentation Draft report with electronic copy Opportunities for feedback Confidentiality Monitoring of plan of action to address weaknesses in GAVI Annual Progress Report

GAVI DQA experience Of the 52* GAVI-eligible countries that applied for ISS funding : Number of Countries Presented for review 43 Passed Under review * 9 countries did not require a DQA as ISS expenditure was below the minimum needed for review 39 4

Innovative: improving MIS

DQA strengths Innovative Management, diagnostic and capacity building tool –DQS – self assessment tool developed to help countries diagnose and address data weaknesses Exposed need to improve skills in management of health data Induced improvements in repeat DQA countries Enhances donor confidence and contributes to transparency and accountability

Limitations of DQA Expensive & labour intensive Imprecision of verification factor –Wide confidence intervals Unable to capture verification of vaccination at point of delivery

Future opportunities for DQA Major evaluation underway to examine methods, sources scope Will review best practice and decision model Opportunities for expanding DQA to include other health system metrics Capacity building to enable local DQA or peer review process