Parallel Session on Monitoring and Evaluation to Inform Action and Set Direction fro Research Five -Year (5YE) Evaluation of the Global Fund Tuesday, 18 November 2008 11:00 - Meeting Room Wa Kamissoko 2008 Global Ministerial Forum on Research for Health – Bamako, Mali

Burkina Faso Team

Scope and Scale of 5YE in Burkina Faso National Health Accounts (2003-2006) National record reviews (HIV, Tb, Malaria, General) Secondary analysis of national surveys (HIV, Tb, Malaria, General) Primary data collection: DCA Facility-based surveys Community-based surveys CBO interventions inventory The National Reference Group for NHA and 2 research institutions selected (IRSS, GREFSaD) 13 Districts selected, 1 per health region, 3 urban, 10 rural, with high (6), medium (3) , & low (4) exposition to support/interventions Meda N, Soudré R, Simaga F, Tiendrebeogo AJ, on behalf of MACRO/WHO/ Burkina Faso 5YE Team

Values of 5YE – Linkages with Research Evidence of poor quality of the routine health information system for the monitoring of VCT, ARV procurement & treatment, PMTCT, etc. Need of technical support (tools, training, archiving) Establish a research-based observatory of universal access Low access to PMTCT services (only 14%): Use coverage of target population to follow the programme instead of number of facilities with PMTCT Introduce in round 9 grant operational research on barriers and facilitators to access PMTCT services by comparing good and bad performers health facilities Rising incidence of TB and malaria 1998-2007 Better detection rate of TB? Better surveillance of malaria? Meda N, Soudré R, Simaga F, Tiendrebeogo AJ, on behalf of MACRO/WHO/ Burkina Faso 5YE Team

Gaps in Linking Evaluation Outcomes with Research – How to Address Them? In-depth analysis of the huge amount of data sets gathered not yet occurred (only imposed indicators have been computed) In-depth description and analysis of the impact on health system indicators not yet done Distribution by disease, coverage, and quality of CBO devoted to the fight against HIV, Tb & Malaria unclear? How to Address Them? Support for further analysis of datasets and publications Study the added value of CBO in the fight against HIV, Tb & Malaria? Meda N, Soudré R, Simaga F, Tiendrebeogo AJ, on behalf of MACRO/WHO/ Burkina Faso 5YE Team

Specific Recommendations – Enhance the Link between M&E and Research Data sets gathered need further analysis to properly answer the basic evaluation questions. The report of GF5YE in Burkina will provide a solid platform for evidence-based policy-making and advocacy in fund-raising (future GF rounds or other partners) and will raise key research questions to explore Operational research has to be funded to address research questions raised by GF5YE findings (low access to PMTCT, rising incidence of malaria and Tb, effectiveness of ARV treatment, lack of CBO in malaria and TB programmes). Meda N, Soudré R, Simaga F, Tiendrebeogo AJ, on behalf of MACRO/WHO/ Burkina Faso 5YE Team

Acknowledgements Global Fund to Fight HIV/AIDS, Tb & Malaria WHO country office and headquarter MACRO international consultants Burkina Ministry of Health/Cabinet CCM/Burkina Task Force members SP/CNLS (NAC secretariat) Research groups and their field workers Community and health facilities surveyed Meda N, Soudré R, Simaga F, Tiendrebeogo AJ, on behalf of MACRO/WHO/ Burkina Faso 5YE Team 7

Ethiopia Team

Scope and Scale of 5YE in Ethiopia Trends in HIV, TB, malaria (3Ds) over past 5 years (since GF’s inception) Not restricted to GF inputs Triangulation across data sources wider health systems Building analytical capacity Using data from multiple sources Modelling to fill data gaps Strengthen country systems for future evaluation Strengthening of specific data sources, collecting baseline information for future M&E Amha Kebede and Dereje Alemayehu on behalf of Ethiopia 5YE Team 9

Values of 5YE – Linkages with Research Capacity building in data collection, assessment and analysis A data depository to facilitate future evaluation A model impact platform to help predict impact based on existing data in country Evidence to support a sustainability plan to strengthen HIS, M&E and quality of services (both at national and district level Trends in disease burden over recent years (or at least established baseline levels) to assess progress toward the MDGs Linkages between health financing, availability of services/intervention and utilization of these services/intervention at district level Assessment of system effects and quality of services in districts according to the level of investment Amha Kebede and Dereje Alemayehu on behalf of Ethiopia 5YE Team 10

Gaps in Linking Evaluation Outcomes with Research – How to Address Them? Lack of mortality data (functioning vital registration system) Limited impact of PMTCT investments Assessing survival rates and quality of life for ART patients Sero-prevalence survey of high-risk populations for HIV Impact of PMTCT program Trends in welfare status of AIDS orphans Population-based data on the prevalence and incidence of TB Trends in STI prevalence; STI surveillance Study impact on human resource capacity as possible bottleneck to service provision Amha Kebede and Dereje Alemayehu on behalf of Ethiopia 5YE Team 11

Specific Recommendations – Enhance the Link between M&E and Research Prioritization needs Avoid redundant efforts (eg World Bank, Resource Mapping, PHRPlus study, CGD study) Avoid multiplicity of implementing organizations Under-utilization of potential of CBOs, including FBOs Assessment of HMIS data quality VCT, PMTCT, ART Follow up of lost-to-treatment cases ART and DOTS Use both qualitative and quantitative methods Conduct service utilization survey Amha Kebede and Dereje Alemayehu on behalf of Ethiopia 5YE Team 12

Malawi Team

Scope and Scale of 5YE in Malawi The main sources for national-level data for both the Primary and Secondary Data Analysis Countries are from existing: Funding records of major national health accounts National service statistics for the three diseases National facility surveys National household surveys ANC surveillance data Other data sources that would inform the evaluation Damson Kathyola and Biziwick Mwale on behalf of Malawi 5YE Team

Scope and Scale of 5YE in Malawi Primary data or district assessments in nine districts Health Facility Survey: rapid assessment of public health facilities Hospital record review: All private and public health facilities within the nine districts were included Patient follow-up study: ART: A sample of patients from a retrospective adults enrolled in ART Information on demographic and clinical characteristics of patients Collected information of characteristics of patients in terms of facilities available for follow-up and diagnosis Challenges: Reluctance of ART clinics to release information especially in districts whose clinics are predominantly under the administration of NGOs Planned to extract data for 1800 but because of the above challenge only managed fro 1200 patients TB: Purposive sampling of all defaulters and patients who did not submit sputum after finalising treatment Interviewed defaulters Collected specimen for other patients Quality control: training of the researchers was undertaken for three days Damson Kathyola and Biziwick Mwale on behalf of Malawi 5YE Team

Values of 5YE – Linkages with Research Although there was a national taskforce, development of indicators and tools was centralised at Consortium level This was a lost opportunity to build and strengthen in-country capacity on M&E Challenges with the Economic/financial component evaluation; Research capacity strengthening within the HIV/AIDS and health sectors on health economics and financial assessment skills There were gaps in information/data from the health centre or national records and lack of a culture at facility level of keeping comprehensive health records e.g. patient registers and other records at facility level- missing/incomplete data Capacity building of the Human resources responsible for M&E within MoH and HIV/AIDS players in M&E to ensure quality Coordination of M&E within the HIV/AIDS sector is a big challenge;- this affects the quality, consistency and comparability of data Damson Kathyola and Biziwick Mwale on behalf of Malawi 5YE Team 16

Values of 5YE – Linkages with Research Although the finance and resource tracking provides an overview of the expenditures on TB, HIV/AIDS and malaria, there is need to undertake, cost-benefit and cost-effectiveness analysis of different interventions Understand which interventions are more effective Economic benefits of the interventions implemented Different designs for different surveys created challenges in comparisons Harmonisation of the conduct of surveys to maximise resources Proper legal frameworks on the conduct of surveys Qualitative assessment of the underlying factors leading to observations in outcomes and functionality of the systems Damson Kathyola and Biziwick Mwale on behalf of Malawi 5YE Team 17

Global perspective: M&E and Research Capacity building initiatives at Global and regional levels should be based on understanding of the gaps in the M&E systems and availability of data for policy decision within the developing countries There is also need to not only build the capacity of the technical staff but also raising awareness among policy makers at global, regional and nation levels on the role of research and M&E in policy formulation to attract political will and support Global plans on diseases and targets should also encompass strategies for strengthening M& E systems to enhance the evaluation of the impact of the interventions Damson Kathyola and Biziwick Mwale on behalf of Malawi 5YE Team 18

Zambia Team

Scope and Scale of 5YE in Zambia Measure the impact of recent interventions or establish a baseline of recent interventions against Malaria, HIV/AIDS and TB Assess the disruption or benefits for other diseases interventions and health care programmes of the scaled-up interventions against the three diseases Develop institutional capacity to utilise information from the Monitoring and Evaluation system for strategic decision making Work still on-going with phase of health facilities records review in three districts with fairly good records and art and PMTCT from computerised records of all districts compiled by national implementing partners Christopher Simoonga, Kumbutso Dzekedzeke and Osward Mulenga on behalf of Zambia 5YE Team 20

Values of 5YE – Linkages with Research Some findings in the 5YE partially answer some research questions but largely bring them out: Just what led to the sharp reductions in child mortality levels ? It should be many factors but it’s not so clear which was dominant among nets, exclusive breastfeeding, better parenting from the improving education of mothers and health education or PMTCT. Answers could be used to focus in cost-effective areas in order to achieve the MDGs in this area. Coverage of interventions did not translate into the same impacts. Are there some unseen factors or agents of interventions? Shouldn’t there be some focus on some research for best health intervention management models? Christopher Simoonga, Kumbutso Dzekedzeke and Osward Mulenga on behalf of Zambia 5YE Team 21

Gaps in Linking Evaluation Outcomes with Research – How to Address Them? There are two levels at which the success of interventions can be assessed Impact (done in 5YE) Diseases burden and it’s changes (Could not be done in the 5YE) In order to assess the changes in the diseases burden and causes of death, at least a sample of facilities should capture individual data over a long period about initial assessment-investigation-treatment-outcome. Currently, pre-coded, unlinked and mostly uni-variate data in the HMIS makes it difficult to do this. Christopher Simoonga, Kumbutso Dzekedzeke and Osward Mulenga on behalf of Zambia 5YE Team 22

Specific Recommendations – Enhance the Link between M&E and Research Expand and improve the data and statistical skills in Health sector. Small establishment overwhelmed with routine compilation of data leaving little time to use the data to answer evaluation and research questions. Health sector should strengthen relationships with local research institutions. Deliberate policy to task such agencies to analyse information would make-up for the shortage of skills and time in health sector agencies Benchmarks linked to incentives for health sector service delivery managers Clear annual targets couched with research and evaluations all the way to the health post Data collection, compilation and transmission to the centre should be a major benchmark for managers at different levels Christopher Simoonga, Kumbutso Dzekedzeke and Osward Mulenga on behalf of Zambia 5YE Team 23