1. How frameworks guide program monitoring and evaluation
M&E Frameworks
How frameworks guide program monitoring and evaluation

2. Learning Objectives
Describe function of Conceptual, Logical and Results frameworks
Apply Proximate Determinants Framework to M&E Planning
Use logical frameworks to guide M&E of HIV programs
Apply achievements to results frameworks

3. Session Overview Function of frameworks Conceptual Frameworks
Proximate Determinants
Logic Frameworks
PMTCT program implementation, GFATM
Results Frameworks
PEPFAR
GFATM M&E Reporting Framework
Activity
Developing a logic framework for an HIV program

4. Discussion
Why do we use frameworks in M&E?

5. Functions of Frameworks for M&E
Roadmap to program planning, monitoring and evaluation
Delineate clear pathways to program goals & objectives
Define relationships between program inputs, processes, outputs, and outcomes
Describe how program factors interact with external context (environmental factors)
Lead to sound implementation and good M&E plans

6. Types of Frameworks - Conceptual
Definition
Illustrate relationships among all factors (systemic, organizational, individual) that influence program operation & successful achievement of goals
Purpose
Show how program components will operate to influence outcomes within wider context
Guide identification of indicators
Guide impact analysis (causal pathways)

7. Conceptual Frameworks
Makes explicit connections among relevant contextual (environmental) factors and your program
Helps clarify the “why” and “how” questions of program operation and design:
Assumptions that underlie chosen activities
Factors activities are expected to affect
Causal linkages leading to achievement of program objectives

8. Conceptual Framework Examples: Proximate Determinants
Premise of “proximate determinants” frameworks:
Factors leading to health or demographic outcome operate through an ordered path of determinants
Determinants grouped in proximity to outcome
Underlying (socioeconomic) factors farthest away, biological closest
“Proximate” determinants take intermediate position between underlying and biological
Proximate determinants are affected by underlying factors & affect the biological determinants
Proximate determinants are what most programs seek to influence

9. Proximate Determinants Example: HIV (STI) Incidence1
Underlying determinants
Socioeconomic and socio-cultural context (personal and community)
Health programs environment: VCT, ARV, STD control, condom promotion, IEC
Proximate determinants (interventions)
Partner acquisition, mixing patterns, concurrency, abstinence
Condom use, STI present, risky practices, STI treatment
Treatment
Biological Determinants
Rate of contact between susceptible and infected
Efficiency of transmission
Duration of activity
Outcomes: HIV incidence/mortality, STI incidence
1Boerma and Weir, 2005
1 Boerma and Weir, 2005

10. Proximate Determinants Framework for HIV/STI1
Underlying
Determinants
Proximate
Biological
Health
Outcome
Demog.
Context
Sociocultural
Socioeconomic
Demographic
Interventions
VCT
STI control
Condom prom.
IEC
Blood safety
Safe injections
Partner acquis.
Mixing patterns
Concurrency
IDU
Condom use
Other STI
Type of sex
Needle safety
ARV Treatment
STI Treatment
Exposure of
susceptible to
infected
Efficacy of
transmission
per contact
Duration of
infectivity
HIV
infection
Mortality
Disease
Think about where your programs fit and what proximate determinants you are trying to change
1Boerma and Weir, 2005
1 Boerma and Weir, 2005

11. Ecological Framework: Links between violence against women and HIV1
Direct transmission
Rape
Gender & relationship inequality
Rape, child sex abuse & IPV
Psychological Distress:
Chronic Anxiety
Depression
PTSD
Substance use
More risky sex:
More partners
Concurrency
Transact. Sex
Sex while intox.
HIV
Indirect transmission
More risky male partners:
More controlling & violent masculinities
More sexual risk taking
More likely HIV/STIs
Reduced protective powers:
More acquiescent femininities
More frequent sex
Less condom use
1 WHO 2010, Addressing violence against women
and HIV AIDS: What works?

12. Types of Frameworks -Logic
Definition
Pictorial diagram that summarizes essential program information by describing its main elements (inputs, activities, outputs, outcomes) & how they work together to affect a desired health outcome
Purpose
Make intended program outcomes explicit
Show internal logic of program
Reveal assumptions of how program leads to outcomes
Guide indicator selection
Promote communication among stakeholders
Guide impact evaluation
Source: CDC Evaluation Guidance Handbook at:

13. Logic Frameworks - Components
Problem statement
The “why” of program implementation
Statement what puts a population at risk for HIV
Inputs
Program resources like staff, curricula, money
Activities
What the program does like outreach, training, testing
Outputs
Deliverables of program like people trained or tested
Outcomes
Program results like change in knowledge, service use
Impact
Long term change in health status like decreased HIV incidence
Source: CDC Evaluation Guidance Handbook at:

14. Logic Framework Example: PMTCT Program1
Problem Statement
HIV rates rising in pregnant women & infants in Thailand. Transmission risk from mom to child significant during pregnancy & delivery, with additional risk pp via breastfeeding
Inputs
Staff, funding from CDC GAP & gov’t, PMTCT counseling/protocol, HIV test kits, ARV drugs, breast milk substitutes
Activities
ANC services include counseling, testing & referral & referral, distribute ARV prophylaxis, provide infant feeding counseling & support
1CDC M&E Guide, Version 1, Appendix 1

15. Logic Framework Example: PMTCT Program, cont’d.1
Outputs (per quarter)
500 women participate in a 2 hr. ANC session
250 of these will get HIV test results
All (of 500) who test positive will be referred for ARV prophylaxis, infant feeding C&S
Half of all positive women will receive prophylaxis and infant feeding counseling and breast milks substitutes
Outcomes
Increase in access to HIV testing
Increase in knowledge of status
Increase in knowledge about prophylaxis
Increase in ARV prophylaxis use by pregnant HIV+
Increase in rapid infant weaning/use breast milk substitutes
Impact: ↓ HIV incidence in infants, ↓ morbidity/mortality among people living with HIV/AIDS
1CDC M&E Guide, Version 1, Appendix 1

16. Developing a Logic Model1
Input from all levels of stakeholders
Program staff, program participants and others to analyze intended program outcomes
A good problem statement leads to a good logic model
Analyze the problem, program resources, desired outputs, outcomes and impact
Design activities to achieve outputs and outcomes
Usually expressed in diagram or table
1CDC, ETA Evaluation Briefs, Logic models, No. 2, Dec. 2005

17. Types of Frameworks – Results1
USAID funded projects, like a logic framework
Definition
A tabular/graphic description of a program’s strategy for achieving a specific strategic objective (SO) through intermediate results, with a narrative supplement.
Purpose
Like a logic framework, planning & management tool for programs
Specifies out exactly how, through a set of intermediate results, the program SO will be achieved
Provides a basis of consensus for all program stakeholders
Conveys the development hypothesis—plausible linkages between the intermediate results and SO
1USAID, Performance Monitoring and Evaluation TIPS: Building a Results Framework, PN-ACA-947

18. Results Frameworks - Components1
Strategic Objective
The ultimate end point of the Results Framework—the goal of the program
“Standard by which the operational unit (performance) is willing to be judged”
Appropriate level of ambition means the SO should be achievable, clear and measurable
Intermediate results
Achievable in the short term, lower level than SO
An intermediate step leading to achieving the SO
Lower level of impact than the SO; cumulative effect leads to the SO
1USAID, Performance Monitoring and Evaluation TIPS: Building a Results Framework, PN-ACA-947

19. Results Frameworks - Components1
Clarification of causal linkages
Narrative that shows how intermediate results are linked to the SO
Links can be between results which then leads to SO
Critical Assumptions
Set of conditions necessary under which the development hypothesis (strategy for achieving the SO) will hold true
Not under project/USAID control, but will affect SO/results
1USAID, Performance Monitoring and Evaluation TIPS: Building a Results Framework, PN-ACA-947

20. Results Framework Example – PEPFAR funded ART Program
SO: Utilization of ART services
IR-1: Availability of quality
services
IR-2: Demand for services
IR-1.1: Increase ART sites
IR-2.1: Increase knowledge of
ART
IR-1.2: Supply sites with ARVs
IR-2.2: Increase referral from VCT
IR-1.3: Training for providers

21. Developing a Results Framework
Set an appropriate strategic objective
Identify intermediate results
Clarify causal linkages between results
Identify critical assumptions
Spend time on the narrative portion so that all stakeholders understand
What is trying to be achieved
How to report verifiable achievements

22. GFATM M&E Framework1 Key Principles Small set of indicators
Country ownership, harmonization, consistency, balance between routine & survey data, avoid duplication of efforts
Small set of indicators
Focus on multi-agency M&E Toolkit (
12-18 indicators per grant
Relies on UNAIDS MERG documents/indicators
No “Global Fund” indicators (country ownership)
Mix of short- & long-term indicators
Quarterly or 6-monthly for some
15-18months for others, important for next Phase
1The Global Fund Monitoring and Evaluation KEY MESSAGES,

23. GFATM M&E Framework, contd.1
Avoid duplication
Use existing in-country indicators, data systems, planned surveys
Strengthen existing systems and budget for data gaps
Indicator targets need to match grant proposals
Existing tools for performance evaluation
M&E Toolkit, M&E Systems Strengthening Tool, Data Quality Assessment Tool
Tools developed for GFATM grants, but applicable to many contexts, as we’ll see
1The Global Fund Monitoring and Evaluation KEY MESSAGES,

24. New Donor Demands: Gender
PEPFAR, UNAIDS, GFATM, WB require gender integration into programs
Addressing gender in meaningful way = measureable
All agencies have strategies to address gender equality
Many resources out there (we’ll examine some at the end of the week)

25. Activity: Developing a Logic Framework
Assemble workshop group-work members
Use provided CDC-GAP activity sheet:
“Logic Model Worksheet for Planned Implementation and Outcomes”
Take about an hour to complete sheet
Resulting logic models to be presented back (5 minutes for each group) by group representative
This draft logic model will guide your group work and form the basis for indicator selection; you will have the opportunity to revise based on feedback

26. MEASURE Evaluation is a MEASURE project funded by the
U.S. Agency for International Development and implemented by
the Carolina Population Center at the University of North Carolina
at Chapel Hill in partnership with Futures Group International,
ICF Macro, John Snow, Inc., Management Sciences for Health,
and Tulane University. Views expressed in this presentation do not
necessarily reflect the views of USAID or the U.S. Government.
MEASURE Evaluation is the USAID Global Health Bureau's
primary vehicle for supporting improvements in monitoring and
evaluation in population, health and nutrition worldwide.