We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byMollie Hoo
Modified over 2 years ago
IPDET Module 11: Evaluating Complex Interventions
IPDET © Introduction Big Picture Views of Development Evaluation Joint Evaluations Country Program Evaluations Sector Program Evaluations Thematic Evaluations Evaluation of Global and Regional Partnership Programs
IPDET © Viewing the Big Picture Reorientation of focus to overall experience and impact Looking beyond one intervention to the effects of all/multiple interventions on: –the entire country, region, or world –one sector –one theme, such as gender –global or regional public goods in one sector or theme (biodiversity)
IPDET © More Comprehensive Agenda Increasingly addressing: –country policy reforms –County or regional capacity building –global concerns Evaluation must expand to: –reorient the focus of evaluation from the project level to the country, thematic, or global levels –seek replicability at a higher level and applicability at the system level
Joint Evaluations Evaluations in which different donors and/or partners participate –donor + donor –donor + donor country –multidonor + multipartner –partner + partner IPDET ©
Planning and Managing Allow sufficient time at the beginning of the process to develop and agree upon process and framework Management systems –simple: meet regularly and share, all equally involved, one is leader –delegate management to one organization, others review key outputs –two-tier: steering committee and smaller management group IPDET ©
Agreements for Joint Evaluations Ground rules for managing the evaluation Terms of Reference (TOR) for evaluation team Means of selecting the evaluation team Budgeting, costing, and financing Reporting findings, including disclosure rules IPDET ©
8 Country Program Evaluations Focuses on an organizations entire aid program to a country Largely a normative study that compares what is being done to what was planned
IPDET © Other Uses for Country Program Evaluations Assess the strategic relevance of the country assistance program relative to the countrys needs Identify the success and failures in different sectors or approaches and identify the factors contributing to performance Assess the effectiveness of the donors aid to a given country
IPDET © Focus of Country Program Evaluations DAC criteria of impact, relevance, or efficiency –impact evaluations can be resource intensive –relevance and efficiency can be less resource intensive (fewer issues)
IPDET © Challenges to Country Program Evaluations Lack of coherent goals in place If funded by several sources, there may be confusion, unnecessary duplication, and different information kept for each intervention No mapping of in-country assistance makes it difficult to know what others are doing Program and/or government officials may be fearful of consequences of the evaluation-- may threatened morale and commitment
IPDET © Recommendations for Country Program Evaluations A greater proportion of evaluations should be undertaken jointly, with full and active participation of the aid recipients and other partners Developing countries should be encouraged to show greater initiative in taking the lead in planning, coordinating, and scheduling evaluations Developing countries should be supported to build their institutional capacity for initiating and leading joint evaluations (continued on next slide)
IPDET © Recommendations for Country Program Evaluations (cont.) Better coordination and knowledge sharing is needed amongst the various partners within aid recipient countries. National M&E networks and professional associations need to be built and expanded When a large joint evaluation is undertaken with the participation of several developing countries, the developing countries should be facilitated to meet together to coordinate their views and inputs.
IPDET © TOR for Country Program Evaluations Need clearly defined TOR –clearly stated purpose, evaluation criteria, and way in which the findings will be used –clearly specified goals of the country program –clearly specified reporting, dissemination, and follow-up procedures
IPDET © Sector Program Evaluations Evaluations on major program sectors A sector covers a specific area activities, such as: education, health, industry, housing, transportation, or agriculture Focus more on questions of institutional performance, processes, changes, and interrelationships An evaluation of a variety of aid actions all of which are located in the same sector, either in one country or cross-country
IPDET © Challenges of Sector Program Evaluations Look at many projects with different objectives and different donors therefore more complex Key issues: –donors concern is to improve the delivery of aid and accountability –partner institutions concern is with improving development in the sector Need to be coordinated as joint evaluations
IPDET © Thematic Evaluations Evaluate selected aspects or themes in a number of development activities Themes emerge from policy statements Look at different kinds of evaluative information and extract aggregate information from these sources
IPDET © Gender in Development: A Thematic Approach Recognizing the complexity of womens and mens social roles Bring a gender focus explicitly into the design of the evaluation Policy-making, planning, and investments must become gender- sensitive
IPDET © Gender in Development Evaluation Allows evaluators to look at many development activities and their effect on gender development Most evaluations do not systematically examine different impacts for men and women Evaluations addressing the theme of gender are a way to look at the role of gender in development
IPDET © Importance of Gender in Development Evaluation Gender inequality reduces economic growth and limits access to public services Because men and women have different economic roles and responsibilities they: –experience poverty differently –have different priorities –face different constraints –are effected differently by development programs
IPDET © Gender Sensitivity and Tools Many tools used in evaluation are not gender sensitive –surveys collect information from the household head who is usually a man –when women are interviewed, other family members may be present, women may not speak freely –women may only speak the local language –women may not attend community meetings, be allowed to speak, or may be expected to agree with male relatives –many studies may use male interviewers, who may not be able to speak to women
IPDET © Data and Gender Ensure data collection methods generate information on both women and men, and that key gender issues are addressed Ensure information is collected about, and from, different household members – not just the household head Use gender-inclusive methods to collect data (continued on next slide)
IPDET © Data and Gender (cont.) Ensure research team includes a balance of men and women Ensure stakeholders are consulted during design, analysis, and dissemination of the evaluation and that consultations include groups that represent both men and women
Evaluation of Global and Regional Partnership Program (GRPP) An increasingly important modality for channeling and delivering development assistance Most are specific to a sector or theme Programmatic partnerships in which: –partners contribute and pool resources to achieve agreed upon objectives over time –are global, regional, or multi-country in scope –partners establish a new organization with a governance structure and management unit –often focus on global public goods IPDET ©
Examples of GRPPs Global Programs –Consultative Group for International Agricultural Research (provides research and improved seed on open-source basis) –Global Environmental Facility (supports conservation of biodiversity and actions to assess and mitigate climate change impacts) Regional Programs -- Nile Basin Initiative (manages joint resources) --West Africa HIV/AIDS and Transport Program (manages joint strategy to prevent spread of communicable disease at border transport points ) IPDET ©
Complexities of GRPP Evaluation Partnership dimension and the need to consider different stakeholder perspectives; need to assess governance mechanisms Open-ended time-frame: Objectives and strategies evolve over time, giving rise to medium-term issues like devolution, or exit, which may need evaluation coverage Cost are incurred and benefits accrued at different levels (results chain complex) (continued on next slide) IPDET ©
Complexities of GRPP Evaluation (cont.) Global and regional dimension means there is a divergence between costs and benefits (evaluation of cost-effectiveness becomes complex) Because of open-ended and longer timeframe, evaluators are often asked to consider progress toward complex medium-term strategic goals IPDET ©
IPDET © Concluding Comments Development evaluators face a unique set of complex economic, political, and social factors as they strive to conduct high quality evaluations that meet the needs of diverse stakeholders These evaluations need to be well-designed and conducted in an ethical manner that is sensitive to local conditions, development issues (i.e., gender and poverty reduction)
A Final Note…. IPDET © 2009 Sometimes the world puts a spin on life. When our equilibrium returns to us, we understand more because we've seen the whole picture. --Davis Barton 29 Questions?
Slide 1 Communications Strategy Day 2. Slide 2 Communications Strategy Planning Communications Strategy Planning.
1 Project Cycle Management and Statistics Module 5.
ActionDescription 1Decisions about planning and managing the coast are governed by general legal instruments. 2Sectoral stakeholders meet on an ad hoc.
Changing the way the New Zealand Aid Programme monitors and evaluates its Aid Ingrid van Aalst Principal Evaluation Manager Development Strategy & Effectiveness.
1 Agenda item 4: Work modalities of the revised ISDR system to support the implementation of Hyogo Framework- Elements to be reviewed in groups- & prepare.
1 European Community Development Policy. 2 GENERAL CONTEXT world-wide.. Trade and investment liberalisation Technological revolution - Information Society.
1 Child Rights Toolkit Comprehensive Toolkit To Address Children's Rights In Development & Humanitarian Cooperation And Government Programming.
Harnessing a multi-stakeholder platform for improved land governance in Malawi Ivy Luhanga – Principal Secretary, Paul Jere – Land Governance Consultant,
EU-Regional Policy Structural actions 1 GROWING EVALUATION CAPACITY THE MID TERM EVALUATION IN OBJECTIVE 1 AND 2 REGIONS 8 OCTOBER 2004.
1. 2 Why are Result & Impact Indicators Needed? To better understand the positive/negative results of EC aid. The main questions are: 1.What change is.
Delivering as One UN Albania October 2009 – Kigali.
Joint presentation by respective units in DGs AGRI, EMPL and REGIO IPA Components III, IV and V: Conditions for successful preparation and absorption of.
Role of CSOs in monitoring Policies and Progress on MDGs. Elijah Wachira UNDP RSC.
Multilateral and bilateral development financing mechanisms that integrate climate change and key issues in making these programmes more effective Phil.
OECD Work on Trade Trade and Agriculture Directorate Paris 2008.
Delivering as One Viet Nam Country-led Evaluation Kigali, 20 October 2009.
Tools and Methodologies in Assessing Technology Needs: An Overview Contact: Prof. Zou Ji Dept. of Environmental Economics and Management Renmin University.
Scaling up the global initiative on the implementation of the SNA and supporting statistics Meeting on Scaling up the coordination and resources for the.
CIDAs Aid Effectiveness Agenda October Canadian aid program CIDA is the lead agency for development assistance The International Assistance Envelope.
Michalis Adamantiadis Transport Policy Adviser, SSATP SSATP Capacity Development Strategy Annual Meeting, December 2012.
1 ACHIEVING THE MILLENNIUM DEVELOPMENT GOALS IN KENYA AN OVERVIEW OF THE MDG PROCESS IN KENYA Mr. George Anyango Ministry of Planning & National Development.
ClimDev-Africa Program & African Climate Policy Center (ACPC) AMCEN Special Session Side Event 27 May 2009.
Lessons from Ethiopia’s Productive Safety Net Programme & Aid Effectiveness.
Overview of Performance Measurement. Learning Objectives By the end of the module, you will be able to: Describe what performance measurement is, and.
The Implementation Structure DG AGRI, October 2005 EU Rural Development.
1 ESA/STAT/AC.219/8 Region-wide Programme to Improve Vital Statistics and Civil Registration Systems prepared by: Margarita F Guerrero, Ph D Regional Adviser.
Australia’s aid for trade approaches Presentation by Henni Arup Multilateral Aid for Trade Section Department of Foreign Affairs and Trade.
European Commission Joint Evaluation Unit common to EuropeAid, Relex and Development Methodology for Evaluation of Budget support operations at Country.
Revision N° 13ICAO Safety Management Systems (SMS) Course06/05/09 Module N° 7 – Introduction to SMS.
FAO Turkey Partnership Programme (FTPP) FAO Final Evaluation of the FTPP Summary for FTPP Programming Meeting, 14 December
Harmonized support to scaling up the national AIDS response Ini Huijts 7 th June 2006 ODI meeting, London.
SADC Course in Statistics Objectives and analysis Module B2, Session 14.
Developing Consensus Principles and Standards for Evaluating Global and Regional Partnership Programs (GRPPs) Progress Report to the Fifth Meeting of the.
Gender and Development Effectiveness. Entry points for Tanzania? DPG Main, 8 May 2012 Anna Collins-Falk, Representative, UN Women on behalf of DPG Gender.
PRESENTATION AT THE WORKSHOP ON LDC PRIORITY NEEDS FOR TECHNICAL AND FINANCIAL COOPERATION 29 OCTOBER 2009 GENEVA Sari Laaksonen Coordinator, Integrated.
Consultant Advance Research Team. Outline UNDERSTANDING M&E DATA NEEDS PEOPLE, PARTNERSHIP AND PLANNING 1.Organizational structures with HIV M&E functions.
11 Scaling Up World Bank Group Engagement with Civil Society: A Strategic Priorities Paper Civil Society Team EXTIA.
Expected Learning Objectives Participants should understand the following: The concepts of ‘gender’ and ‘sex’. The term ‘gender equality.’ The importance.
Denmark-Tanzania Country Policy Country Policy to be implemented through Country Programme ($400m) Overall objective remains fighting.
3-Jun-14 UNDP-SNV Focal Points Meeting Nairobi !6-17 May SNV and UNDP ACTIVITY AGREEMENT 4 Rwanda.
1 The new ESF Investing in your Future -
1 Ultimate Outcomes Settlement Program Logic Model Immediate Outcomes Program Components CIC Strategic Outcome 3 Outputs Intermediate Outcomes Policy Development,
Independent Evaluation Group World Bank November 11, 2010 Evaluation of Bank Support for Gender and Development.
1 Project Appraisal Module 5 Session 6. 2 Summary This session will introduce dimensions of project appraisal, including: issues of social acceptability/desirability,
ESPON Seminar 15 November 2006 in Espoo, Finland Review of the ESPON 2006 and lessons learned for the ESPON 2013 Programme Thiemo W. Eser, ESPON Managing.
Jean-Noel Guillossou Program Manager, SSATP SSATP Institutional Framework Annual Meeting, December 2012.
Module N° 4ICAO State Safety Programme (SSP) Implementation Course 1 Module N° 4 – ICAO SSP framework Revision N° 5ICAO State Safety Programme (SSP) Implementation.
UNDP RBA Workshop on MDG-Based National Development Strategies Module 1: MDG-Based Planning UN Millennium Project February 27-March 3, 2006.
Carlo Carugi, Senior Evaluation Officer GEF Independent Evaluation Office March, 2015 Evaluation in the GEF and Training Module on Terminal Evaluations.
© 2017 SlidePlayer.com Inc. All rights reserved.