1. EVALUATION AU Roadmap Indicators, qualitative data requirements and collection process
Sabelo Mbokazi
Senior Policy Officer HIV/AIDS, TB, Malaria & OID
Department of Social Affairs
African Union Commission

2. Presentation Outline
Part I – AU Roadmap Indicators
Part II – Qualitative Evaluation Criteria

3. Introduction: Aspects of Qualitative Evaluation
Evaluation relating to or based on examination and judgment of the value, quality, or importance of something
Qualitative data collection methods will be employed to answer the WHY? & HOW? on the implementation of the policy framework:
Discovering new meaning and perspectives
Determining the extent and frequency of actions
Categorisation of information
 Qualitative approach will facilitate generation and documentation of new knowledge for utilisation in the posterior

4. The Roadmap: three action pillars1
More diversified, balanced and sustainable financing models 2
Access to medicines – local production and regulatory harmonisation 3
Leadership, governance and oversight for sustainability
Priority actions
Use strategic investment approaches for scale-up of basic programmes
Support communities to claim their rights and participate in governance of the responses
Ensure investments contribute to health system strengthening
Mobilise leadership at all levels to implement the Roadmap
Invest in leading medicines manufacturers – focusing on AIDS, TB and malaria
Lay foundations for a single African regulatory agency
Acquire essential skills through technology transfers and south-south cooperation
Incorporate TRIPS flexibilities and avoid "TRIPS-plus" measures in trade agreements
Develop financial sustainability plans with clear targets
Ensure development partners meet commitments and align with Africa’s priorities
Maximise opportunities to diversify funding sources and increase domestic resource allocation

5. 3. Evaluation focus
AU Roadmap on for Shared Responsibility and Global Solidarity
The key evaluative question that this policy review seeks to answer is:
What changes the implementation of the AU Roadmap has induced in the fight against HIV and AIDS, Tuberculosis, Malaria in the context of the three pillars?:
More diversified, balanced and sustainable financing models
Access to medicines – local production and regulatory harmonisation
Leadership, governance and oversight for sustainability

6. AU Roadmap: Country Level Indicators
Pillar I: Diversified, balanced and sustainable financing models
1. % of donors meeting their funding commitments

7. AU Roadmap: Country Level Indicator
Pillar II: Access to Medicines and Regulatory Harmony
1. 3 % of reduction of tariffs on AIDS, TB and Malaria commodities
2. Extent of accessibility to quality assured generic medicines and other Anti-Malaria commodities as well as Anti-TB
3. Extent of implementing regionally harmonized regulatory guidelines, and joint dossier assessments and inspection of pharmaceutical manufacturing plants
4. Extent of adapting legislation using the AMRH Model Law as a guideline

8. Pillar III: Leadership, Governance and Oversight for Sustainability
1. Proportion of GDP spent on health
2. % NCPI (national commitments and policies instruments) ratings for Strategic Planning, Political Support and Leadership and Human Rights
4. Extent of engagement of national human rights bodies in strategy development and review processes
4. Extent of engagement of national parliaments in review process

9. Qualitative Data Requirements
Definition: The extent to which the objectives of the policy framework are consistent with the requirements, needs and priorities of the target audience (MS)
Relevancy of the policy framework in addressing the identified challenges
Appropriateness of the actions/commitments resulting from the policy instrument
Validity of the objectives of the policy instrument
Relevance

10. Qualitative Data cont …
Definition: The extent to which the objectives of the policy framework have been achieved within the life span of the policy instrument
Effectiveness of the policy instrument in addressing the health challenges on the continent
Achievement of the objectives of the policy documents
Target population reach
Timely implementation
Resource utilization
Effectiveness and efficiency

11. Qualitative Data cont …
Definition: The extent of domestication and broader involvement of all relevant stakeholders
Level of alignment to the national policies and frameworks
Participation of key stakeholders
The extent of productivity of collaborative & partnerships initiatives
Ownership/participation/Partnerships

12. Qualitative Data cont …
Definition: The totality of the effects of concrete (positive and negative) intended and unintended actions directly associated with the policy framework
Impact on the health of the African people
Changes that can be attributed to the policy instrument
Impact (Outcome level)

13. Qualitative Data cont …
Definition: The continuation or longevity of benefits from the implementation of the policy framework after the cessation or expiration of the life span of the policy.
(e.g. To aptly examine sustainability the evaluation may focus on review of financial viability, skills capacity developed (e.g. domestic financing to ensure SRHCS, ARVs uninterrupted availability)
Continuity of interventions resulting from the policy instrument beyond their timelines
Attributed change can be sustained beyond the policy instrument deadlines
Sustainability

14. Key Evaluation considerations
On the main information is composed of Secondary Data – hence qualitative data assists to enrich appreciation of contextual dynamics
To ensure data quality assurance and harmonisation AUC supported by the partners will develop data collection tools for each policy instrument and disseminate
Effort to align the outcome of the health policy review to the CAP goals should be made
Data disaggregation (gender, age, rural/urban)

15. Qualitative data collection plan
How? - Discussion the modalities
Who? – Pledges of support and discussion on sources of information
When? - Time frame

16. I thank you