1. Decision-Making and Strategic Information Workshop on M&E of PHN Programs July 24-August 11, 2006 Addis Ababa

2. Learning Objectives By the end of the session, participants will be able to: Identify basic purpose of monitoring and evaluation (M&E) Understand the components and uses of strategic information Identify key elements in decision-making process List common barriers to using M&E data for decision-making and ways to overcome them

3. Monitoring versus Evaluation MONITORING = Tracking changes in program performance over time EVALUATION = Attributing program outcomes to their causes

4. Illustration of Program Monitoring Program start Program end TIME-> Program indicator

5. Illustration of Program Impact Program start Program end TIME-> Change in program outcome With program Without program Program impact

6. Purpose of Monitoring and Evaluation The purpose of monitoring and evaluation is to measure program effectiveness.

7. What Information Do Decision Makers Need M&E Data to Provide? Process: –Was the program carried out as planned? –How well was it carried out? Results: –Did the expected change occur? –How much change occurred? Impact: –Is the change attributable to the program? –Does the change mean program “success”?

8. STRATEGIC INFORMATION

9. Data Versus Information Can be used interchangeably, but: Data often refers to raw data, or unprocessed information Information usually refers to processed data, or data presented in some sort of context

10. Definition of Strategic Information Monitoring & Evaluation Evidence-based Research Surveillance Management Information System (MIS) Routine Health Information System (RHIS)

11. Components of Strategic Information

12. Uses of Strategic Information Strategic Information is the foundation upon which all planning and program design decisions are based. Strategic information facilitates program improvement, evaluates progress, and ensures policy compliance.

13. Strategic Information & Program Life Cycle ASSESSMENT What is the nature of the (health) problem? EVALUATION How do I know that the strategy is working? How do I judge if the intervention is making a difference? STRATEGIC PLANNING What primary objectives should my program pursue to address this problem? MONITORING How do I know the activities are being implemented as designed? How much does implementation vary from site to site? How can be program become more efficient or effective? DESIGN What strategy, interventions and approaches should my program use to achieve these priorities ? 1 2 3 4 5

14. UNDERSTANDING decision-making

15. What is Decision-Making? The process of choosing from among various alternatives using information

16. What Determines Utilization of SI for Decision-Making? SYSTEMS APPOACH TECHNICAL APPOACH INDIVIDUAL BEHAVIOR POLITICS CULTURE SOCIETY

17. Understanding Decision-Making Decision makers Decisions

18. Understanding Decision-Making Decisionmakers SectorHealthEducationCommercialFunctionPolicyPlanningBudgeting Service delivery AdvocacyEvaluationLevelNationalRegionalLocal

19. Types of Decisions Decisions Policy-making… Work plan development … Resource allocation Budget Human resources Infrastructure

20. Understanding decision-making Why? Who else? How? Decisionmakers Decisions

21. Understanding Decision-Making DecisionmakersDecisions Why? Who else? How? Whatinformation?

22. Use of data and other information for decision- making occurs within a given context.

23. Who is Involved in Decision- Making? Different types of decision makers View activities from different perspectives Have different degrees of understanding of the program Need different information Want different information Need or want information at different levels of complexity Have different intensities of interest

24. Examples: Decision Makers and Information Needs

25. How are Decisions Made? Rational/Scientific Political Routine

26. When Is Decision-Making Not Political? Submitted Answer to AEA Presidental Address Challenge in 1989 by R. Turpin No one cares about the program No one knows about the program No money is at stake No power or authority is at stake No one in the program And, no one in the program is making decisions about the program, or is otherwise involved in, knowledgeable about, or attached to the program

27. Class Activity: Data Use & Decision- Making Why are decisions made in your organization? Who makse decisions in your organization? How are decisions made in your organization? Discuss and present a time when data was used to make decisions... … and a time when other factors outweighed data utilization in decision-making

28. UTILIZATION-FOCUSED EVALUATION

29. Utilization-Focused Evaluation Doing evaluations that are useful and actually used Evaluations are largely unused As are research results New directions in accountability Increasing demand for professional evaluations Need to bring use into practice

30. Standards for Evaluation Utility - serve practical information needs of intended users Feasibility - be realistic, prudent, diplomatic and frugal Propriety - conducted legally, ethically, and with regard to those involved in and affected by the evaluation Accuracy - reveal and convey technically accurate information

31. Class Discussion What are the kinds of situations that pose special challenges to the utilization of M&E data for program decision-making? What factors should be considered in order to foster the utilization of M&E findings by intended users?

32. Increasing Use of M&E Data: Ad-hoc Evaluations Develop realistic recommendations for program improvement Explore multiple uses of study data Continuously remind decision makers of findings & recommendations Share findings & recommendations with broad audiences Assign evaluation staff to assist in implementing recommendation

33. Increasing Use of M&E Data: Outcome Monitoring Provide timely reports Involve program staff in definition of outcome measures and data collection Maintain high face validity of outcome data Demonstrate use of outcome information Repeat outcome measures on regular basis Mandate performance monitoring

34. “Seven Use-Deadly Sins of Evaluators” (per Patton, 1997:54) Act as the primary decision-makers Identify vague, passive audiences as users, instead of real people Target organizers as users, instead of individuals Focus on decisions, instead of decision-makers Assume the evaluation’s funding agency is automatically the primary stakeholder Wait until findings are in to identify intended users and intended uses Distance themselves from people and politics

35. Closing Comments Information must be based on quality data in order to be useful Information must be communicated effectively in order to be useful Information must be used to be good M&E

36. References Patton, Michael Quinn. 1997. Utilization-Focused Evaluation. Thousand Oaks: Sage Publications.