1. Promoting Rational Drug Use in the Community Monitoring and evaluation

2. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 2 Objectives Session on Monitoring and Evaluation  Understand functions of monitoring and evaluation  Define what should be monitored  Formulate evaluation questions  Identify methods  Make a plan for monitoring and evaluation

3. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 3 Place of Monitoring and Evaluation in the Planning Cycle Problem Goal Objective Activity Monitoring Evaluation Monitoring Refers to Activities Evaluation refers to Objectives

4. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 4 Functions of monitoring  See that everything goes according to plan  Find out if there are unexpected difficulties  Adjust plans

5. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 5 Functions of Evaluation  Prove that the intervention worked  Assess if costs were reasonable  Convince others that intervention works  Share experiences - to allow replication

6. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 6 Indicators  Indicators are needed for both monitoring and evaluation  Monitoring requires process indicators  Evaluation requires outcome indicators  Planning to collect indicators is an important part of any plan

7. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 7 What to monitor  Formative research(RAP) done?  Intervention pre-tested?  Activities implemented according to plan?  Are costs in accordance with budget?  Are staff carrying out assigned duties?  Is collaboration with others taking place?

8. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 8 How to monitor  Record keeping  Making reports on activities  Field or supervisory visits  Regular project/programme meetings

9. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 9 What to evaluate?  Preparation  Planning  Implementation  Effect

10. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 10 Preparation  How was the problem identified?  Was formative research done  Who is involved in the intervention  How comprehensive is the intervention (consider the factors which cause the problem)

11. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 11 Planning  What objectives were set?  Which activities were planned?  What target audiences were identified?  Were the interventions pre-tested?  Was a plan for monitoring and evaluation made?

12. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 12 Implementation  Which of planned activities were carried out?  What messages were disseminated?  How many people did the message reach?  Did the target audience pay attention?  Did the target audience understand?  What problems were encountered

13. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 13 Effect  Change in knowledge?  Change in behaviour?  Change in health service quality?  Change in policy?  Improvements in health?  Negative or unexpected effects?

14. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 14 Four evaluation designs  Randomised control design  Quasi experimental design  Time-series design  Pre-post design

15. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 15 Four evaluation designs Randomised control design intervention group receives education è random assignment of intervention and control group; measure change è at the beginning after intervention of the intervention control group receives no education Quasi-experimental design intervention group receives education è specifically selected intervention and control group; measure change è at the beginning after intervention of the intervention control group receives no education Type of designActionMeasuring change

16. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 16 Four evaluation designs Time-series design è measure baseline measure change twice (for example at one + six month intervals) after at the beginning of implement intervention intervention the intervention and ask questions to find out why people changed behaviour Pre-post design è measure baseline measure change at the beginning ofimplement intervention after intervention the intervention and ask questions to find out why people changed behaviour Type of designActionMeasuring change

17. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 17 Define key outcome measures  Review communication objectives  Identify what behaviours are likely to change  Limit the number of outcome measures  But, measure more than one dimension  Choose outcome measures that can be clearly defined and reliably measured

18. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 18 Examples of key outcome measures  The percentage of childhood diarrhea cases treated with antidiarrhoeal medicines  The percentage of total antimalarial sales which included an adequate dosage

19. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 19 Methods  Review of project documents, including monitoring reports  Semi-structured interviews  Short quantitative surveys  Focused weekly illness recalls (change in common health problems)  Structured observations

20. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 20 Problems in proving effects  Contamination  Intervention changes over time  Difficult to measure mix of methods  Unplanned interventions by others  Confounding factors

21. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 21 Summary guidelines  Decide at beginning of an intervention how you will evaluate  Prepare good outcome measures  Evaluate process and effect  Look for short- and long-term change  Encourage participation of target audience  Share successes and failures  Make an evaluation plan

22. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 22 An evaluation plan  Statement of communication objectives  Evaluation questions  Key outcome measures  Evaluation design/methodology  Data collection methods  Plan for data processing and analysis  Plan for dissemination of results

23. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 23 will depend on the complexity of behaviour, its reinforcement in culture, the presence/absence of opposing forces and the resources at your disposal Short term knowledge,skills, awareness Medium term 1-3 yrs behaviour change Longer term <5 yrs Change in health What can be achieved?

24. WHO Monitoring and evaluation Promoting Rational Drug Use in the Community 24 Activity 1: Evaluate a shopkeeper intervention/Uganda  Read the description, page 16  Formulate: Evaluation questions  Select key outcome measures  Advise on study design: quasi- experimental or time-series design?  Propose limited set of data-collection methods