METHODOLOGY FOR THE REVIEW/EVALUATION OF POLICY DOCUMENTS By Kwami DADJI, Health Officer HIV/AIDS, TB, Malaria & OID African Union Commission

Presentation Outline 1.Introduction 2.Objectives of the Evaluation 3.Evaluation questions 4.Methodology

1. Introduction The main health policy documents of the African Union are expiring on The final evaluation of the Abuja Call and for the Maputo Plan of Action will be conducted to assess their level of implementation by the UA Member States. The outcome will be presented to the Specialized Technical Committee on Health in April 2015 and to the AU Assembly of Heads of State and Government in June / July 2015.

2. Objectives General objective To document and assess the outcomes according to the set objectives of the Abuja Call and the Maputo Plan of Action. Specific objectives Assess the relevance (the global context) of the Abuja Call and the Maputo Plan of Action; Assess the coherence of priority areas under the Policy documents (Abuja Call and the Maputo Plan of Actions) and indicators to measure them (nature of the indicators, ease of measurement, tools, methods and means of collection, analysis, etc.); Assess the effectiveness of actions implemented in the various countries in the strategic areas identified by the Policy documents ; Determine the level of achievement of key indicators in the context of monitoring and evaluation;

2. Objectives Specific objectives Assess the adequacy of the main indicators used in the monitoring and evaluation framework with policy documents and national strategies; Analyze the partnership engaged at the country, regional and continental level (type, functionality, advantages and disadvantage); Identify the strengths and weaknesses of the implementation of the Policy documents ; Analyze the implementation of the recommendations of the mid-term reviews; Propose a decision regarding the continuation, reorientation or termination of implementing the Policy documents, taking in account the results and recommendations of the review as well as the African Common position on the Agenda beyond 2015; In case of continuation or redirection, provide consistent recommendations for better implementation of the Abuja Call and the Maputo Plan of Actions

3. Evaluation questions General Question Abuja Call The key question that this final evaluation seeks to answer is: what changes the implementation of the Abuja Call has induced in the fight against HIV and AIDS, Tuberculosis, Malaria and other infectious diseases in the AU Member States? Maputo Plan of Action The key question that this final evaluation seeks to answer is: what changes the implementation of the Maputo Plan of Action has induced in the improvement of the Sexual Reproductive Health and the MNCH in the AU Member States?

3. Evaluation questions Specific questions Specific questions that will guide the overall work of the evaluation are as follows Abuja Call - What is the relevance of the Abuja Call in the fight against the priority diseases? - What is the efficiency in the implementation of the Abuja Call in the 5 African regions? - The expected outcomes were they produced? There were any unexpected outcomes? - What impact(s) in the medium term can be expected from the implementation of the Abuja Call at country, regional and continental levels? -What is the added value of the Abuja Call in the fight against these priority diseases, for example on health financing in Africa and the strengthening of health systems? - Should we continue with the priority areas proposed by the Abuja Call? termination or reorientation?

3. Evaluation questions Specific questions Specific questions that will guide the overall work of the evaluation are as follows Maputo Plan of Action - What is the relevance of the Maputo Plan of Action in the improvement of the Sexual Reproductive Health and the MNCH? - What is the efficiency in the implementation of Maputo Plan of Action in the 5 African regions? - The expected outcomes were they produced? There were any unexpected outcomes? - What impact(s) in the medium term can be expected from the implementation of the Maputo Plan of Action at country, regional and continental levels? -What is the added value of the Maputo Plan of Action in the improvement of the Sexual Reproductive Health and the MNCH and the strengthening of health systems? - Should we continue with the priority areas proposed by the Maputo Plan of Action? termination or reorientation?

4. Methodology Conducted by the Department of Social Affairs of the African Union Commission (AUC) The approach to be adopted is based primarily on the analysis of policy documents and commitment of the Member States, the monitoring and evaluation Plan, activity reports, the mid- term reports and any other documents relating to the implementation

4. Methodology The evaluation approach will thus structured around the following activities: – Technical meeting at the AUC involving key stakeholders, to validate the methodology and tools for data and information collection; – Send data collection tools to members States and to partners for filling and collection of data on indicators. – Invitation of selected Member States to discuss the relevance of selected indicators, measurement and difficulties of providing information in real time; – Analysis of data and information; – Draft the evaluation report; – Validation workshop to finalize report; – Translation and submission of the report to the relevant stakeholders

4. Methodology Sampling After analyzing the mid-term evaluation reports, it will be selected in the five African regions, 10 countries including five which have the highest levels of key indicators and 5 which have the worst levels of the indicators. The countries representatives of the selected countries will be invited to AUC to discuss the main issues related to the implementation of the policy documents in their respective countries, strengths and weaknesses. Moreover, the discussions will also address the challenges of availability and sharing of information on key health indicators to enable monitoring to achieve the Abuja Call.

4. Methodology Data collection tools TechniqueTools Mixed: Quanti - Qualitative Questionnaire for country indicators Qualitative Guide for group discussion with partners Guide for discussion with representatives of countries invited to the meeting to exchange with the AUC or by telephone interview, Skype etc...

4. Methodology Data analysis Data analysis will be done at three levels. The first level will be to analyze the relevance of policy documents from an approach based on the qualitative analysis of the situational analysis of health in Africa and the framework of these documents. Regarding effectiveness in the implementation of the policy documents, the analysis will be based on the measurement of the level reached by the different Member States for the main indicators used in their monitoring and evaluation plan. In a third step, an contained analysis will be made on qualitative information gathered from key stakeholders (partners and representatives of the countries)

I thank you