1. Session 13: Monitoring and Evaluation PubH325 Global Social Marketing Donna Sherard, MPH November 30, 2009

2. Session objectives Understand definition of Monitoring AND Evaluation Understand what both Monitoring and Evaluation measure Know important considerations for R, M/E plans

3. Monitoring AND Evaluation Guess What…? Monitoring and Evaluation are Different!

4. The regular follow up of the implementation of planned activities; The systematic and continuous process of following and keeping pace with indicators in order to ensure that project/program is proceeding according to plan. D.S. Opinion: Poor or nonexistent monitoring is the death knell to adequate funding for communication! What is Monitoring?

5. What does monitoring answer? Are we heading in the right direction? What can be improved in our project? Are we utilizing resources efficiently? Provides updates for stakeholders and donors

6. What is Evaluation? The systematic assessment of effectiveness and efficiency of the project achievements based on the set objectives

7. What does evaluation answer? Have the outcomes/objectives been met? How effective were strategies used to implement project activities? What lessons have been learned from the project? In sum: Did exposure to our intervention result in the desired behavior change or positive change in determinants?

8. Main Types of Evaluation: 1. Process evaluation – assess efficiency and effectiveness of implementation 2. Mid-term evaluation – assess effectiveness and efficiency at project mid-point 3. Impact Evaluation – Measures the extent to which the intervention has caused change in desired direction

9. Ok…Let’s Talk About the Real World

10. Key Questions for M/E (Kotler and Lee) 1. Why are you conducting this measurement? 2. What will you measure? 3. How will you conduct these measurements? 4. When will the measurements be taken? 5. How much will it cost?

11. What will you measure? 1. Output/Process Measure- quantifies marketing activities 2. Outcome Measures- Assessment of TG response to outputs 3. Impact Measures – Measurement of the impact that the changes in behavior have achieved (time, feasibility, control)

12. Conceptual Framework for Condom Use among Male Clients of Female Sex-workers in Karnataka, India Social norms regarding condoms Condom use Condom availability Condom access Subjective norms regarding condoms Condom use outcome expectations Self- efficacy Condom use intention HIV knowledge HIV threat Gender roles Mass media access Mobile employment Alcohol consumption among males Contextual factors Individual factors Community factors Behavioral outcome

13. PSI Log Frame NarrativeIndicatorsMeans of verification Assumptions Goal (Impact): Improved health Health indicatorsMeans of verification Assumption out of project control that could affect outcome Purpose (Outcome): Changed behavior including use of product or service Measures of changed behavior Program Evaluation Output (Outcome): Positive changes in determinants Changes in determinants Program monitoring Activity (Output/Process): The 4 P’s plus research and training Input and resources needed Activity Monitoring

14. Log Frame…Again NarrativeWhat’s MeasuredMeans of verification Assumptions Goal (Impact): Improved health Health indicatorsAssumption out of project control that could affect outcome Outcome Changed behavior including use of product or service Changes in determinants Program Evaluation Output/Process: The 4 P’s plus research and training Input and resources needed Activity Monitoring

15. Data Collection Options Population based surveys (quantitative) Observation surveys – support supervision, ‘mystery client’ Randomized Control Trial

16. Conclusions Consider R,M and E from the very beginning of program planning Get clear about what you want to measure, when and why (Kotler’s questions) Evaluation- final assessment of efforts Monitoring – ongoing measurement of progress

17. Quote for the Day! “Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted” - Albert Einstein