1. 1 Virtual training Panama City, June 2009 Module 8 Monitoring, Evaluation and Learning

2. 2 Photograph by Bob Krist National Geographic Objective The module is intended to provide answers to the following questions: 1. What is the purpose of the evaluation? 2. Who will use the evaluation results? 3. Who will do the evaluation? 4. What evaluation framework is practical? 5. What needs to be monitored and evaluated? 6. What are the steps to develop a self-assessment matrix? 7. How can you use the evaluation to enhance a learning culture that keeps improving your IEA process?

3. 3 Introduction How many of the policy-makers use integrated environmental assessments (IEA) as a trusted resource, and consider it a learning opportunity to improve policies? How do we know whether the assessment is useful and used, rather than just sitting on a shelf?

4. 4 Stages of the national IEA,

5. 5 Monitoring Defined Monitoring is a planned, systematic process that closely follows a course of activities, and compares what is happening with what is expected to happen. Monitoring the IEA process makes sure the assessment meets its goals, while working within the scope of allocated resources (i.e., time, financial, human, informational and technical). Photograph by David Doubilet National Geographic

6. 6 Evaluation Defined Evaluation assesses an achievement against preset criteria. Evaluations can have a variety of purposes, and follow distinct methodologies (process, outcome, performance, etc.). Evaluation of the IEA process determines the extent to which achievements (outputs, outcomes and impacts) are comparable with the originally intended purpose, and what lessons can be learned for the next environmental assessment and management cycle. The evaluation of the process is, first and foremost, a capacity-development opportunity.

7. 7 Compare Monitoring and Evaluation Monitoring 1.Collecting data 2.Sense of progress 3.Present time 4.Attention to details 5.Inspires motivation 6.Occurs continuously 7.Requires management skills 8.Asks: What needs to happen now to achieve our goal Evaluation 1.Assessing data 2.Sense of achievement 3.Past / future time 4.Attention to bigger picture 5.Inspires creativity 6.Occurs intermittently 7.Requires leadership skills 8.Asks: Have we achieved our goal? Can we do better next time?

8. 8 Learning defined An emotional and/or cognitive transformation taking place during information collection and information processing, bringing about behavior change or change in the ability to act differently. Photograph by Sheldon Smith National Geographic

9. 9 Foundation steps for effective Monitoring and Evaluation 1.Identifying your purpose. 2.Identifying your primary users. 3.Deciding whether internal or external evaluators best serve your purpose. Photograph by Norbert Boroviec National Geographic

10. 10 1. Types of Purpose 1.Judgment: Sets clear criteria and standards to judge performance. Can increase credibility of the GEO-process. 2.Improvement: Open-ended evaluation that measures change over time. Often applied to cyclical activities, like the GEO Process. 3.Knowledge-creation: Identifies emerging knowledge and insights. Increases the saliency of the GEO process.

11. 11 2. Users of a IEA type evaluation People who: can revise the GEO process: have the mandate, knowledge and skills; and want to revise the GEO process: have a vested interest in influencing the design and implementation of the GEO process. Photograph by Mick Dees National Geographic

12. 12 Primary Users The primary users of the evaluation may include: –IEA core team (includes policy- makers); –policy and decision-makers who are primary users of the IEA; and –the evaluation team (internal and/or external). Photograph by Olivier Grunewald National Geographic

13. 13 3. Evaluators Evaluators may include: A small internal evaluation task force (including the IEA core team, which is recommended). External evaluators (consultants and internal evaluators of another IEA). A combination of internal and external parties. Photograph by Steven Lewis National Geographic

14. 14 Framework for Monitoring and Evaluating the National IEA Process

15. 15 Possible Measures for Effective Knowledge Management

16. 16 Planning a self assessment Step 1 Identify major issues and monitoring questions, and develop specific measures. Step 2 Identify sources of data and data collection methods. Step 3 Set priorities and frequency of monitoring Photograph by Kazuyoshi Nomachi National Geographic

17. 17 Step 1. Identify Major Issues and Monitoring Questions, and Develop Specific Measures Outcomes Your Change Statement Effective Relationship Management Activities and Outputs Effective Knowledge Management Effective Opportunity Management Timely completion of activities and outputs

18. 18 Outcome-based Measures: Possible Organization of Your Self- Assessment Matrix Key Issues/QuestionsSpecific Measures and Target Data SourceData Collection Method Your Change Statement Effective Relationship Management See Table 4 for guidance

19. 19 Step 2: Identify Sources of Data and Data Collection Methods

20. 20 Step 3: Set priorities and Frequency of Monitoring and Evaluation Ensure that critical indicators are monitored. Establish the frequency of monitoring for each indicator.  Process indicators are needed throughout the IEA.  Progress indicators are needed less frequently, but long after the IEA is complete.

21. 21 Improvement Opportunities Thinking of an IEA as a capacity-building process significantly increases its impact. The more that monitoring and evaluation is treated as an organizational learning opportunity (versus a value judgment), the more effectively the IEA supports improvements in policy making. Photograph by Richard T. Bryant National Geographic

22. 22

23. 23 Actividad: Experiencia previa y limitaciones

24. 24 1.El objetivo de este ejercicio es identificar experiencias previas en M&E y limitaciones que enfrentan las organizaciones ante las tareas de monitoreo y evaluación. 2.En grupos pequeños (3-4 personas), identificar aquellas áreas del proceso de la EAI en las que sus organizaciones podrían enfrentar limitaciones capaces de restringir las tareas de monitoreo y evaluación. 3.Compartir en plenario Ejercicio

25. 25 Actividad: M&E contra la estrategia de impacto

26. 26 Recordar… Step 1 Changes (CS) Step 2 Who? (RM) Step 3 What? (KM) Step 4 How? (OM) Step 5 M&E - Key messages - Issue cycle Other influences on decisions and decision-makers Participants adding to the saliency, legitimacy and credibility

27. 27 Previamente… Considere las etapas de: –Relationship Management (RM) –Knowledge Management (KM) –Opportunities Management (OM) Revisar las secciones 3.3.2-3.3.4, páginas 14-17 del GEO Resource Book.

28. 28 Responder En grupos de 3-4 personas. Adicionalmente a las medidas presentadas en las tablas 2, 3 y 4 del GEO RB (módulo 8), ¿puede sugerir otras? ¿Qué medidas considera más importantes? ¿Qué metas considera más razonables para dichas medidas? Presentación plenario