IAOD Evaluation Section, the Development Agenda (DA) and Development Oriented Activities Julia Flores Marfetan, Senior Evaluator

2 1.Introduction to the Independent Evaluation Function 2.Services provided to the Development Agenda 3.Independent evaluation of the Development Agenda and development oriented activities 4.Types of Evaluations 5.Evaluation Approach for the Development Agenda and development oriented activities IAOD Evaluation Section

Introduction to the Independent Evaluation Function

Evaluation Definition Evaluation is an objective, impartial and systematic assessment to provide useful feedback about an activity, initiative, project, program, strategy, policy, unit, organization or sector, whether financed from regular budget or extra budgetary resources 4 IAOD Evaluation Section

Evaluation in WIPO Where did all start? –Recently no real evaluation activity since 2004 –Evaluation was perceived as a program management function What has IAOD and its Evaluation Section done since 2007? – The WIPO Evaluation Policy since 2007 – The Evaluation Section was set-up by the DG in 2008 within IAOD with a dedicated Evaluation Team – Focusing the evaluation function in their core business “Independent Evaluations” – Independent Review of the PPR process – Validation of the 2008 PPR – Having dedicated evaluation team within the Evaluation Section – 2 post were created – Training to improve existing monitoring and evaluation frameworks – Development of self-evaluation guidelines for program staff – Improving the previous PPR and making it more performance oriented 5 IAOD Evaluation Section What are the Evaluation Section future priorities? –Undertake high quality independent evaluations of development oriented activities in form of country, thematic, strategic or program evaluations –Develop WIPO Evaluation Strategy and Guidelines –Contribute to enhance accountability, learning and transparency by delivering relevant information to our stakeholders specially Member States

6 IAOD Evaluation Section EVALUATION SECTION SERVICES

Services provided to the Development Agenda (DA)

May help Member States in building their capacities to undertake evaluation exercises Including impact evaluations depending on resources provided. This is important for The country focus of the DA and DOA. Design, manage and undertake independent evaluations of the Development Agenda and development oriented activities Other Support Services CORE SERVICES: INDEPENDENT EVALUATIONS Support for capacity building Ensure that monitoring and performance management of the development oriented activities is of good quality Validation 8 IAOD Evaluation Section EVALUATION SECTION SERVICES

Independent Evaluation of Development Agenda (DA) and development oriented activities (DOA)

Independent evaluations contribute towards a UN system identifying ways to improve delivery of development oriented activities to the beneficiaries; overcoming weaknesses and building on strengths from a strong evidence base Independent evaluations assist Organizations to take a deep and detailed look at internal systems and development oriented activities, and examine what practical steps need to be taken to improve them According to the UNEG norms and standards the evaluation function and its products should be independent, credible, and useful for learning and accountability purposes It is central to the credibility and reliability of evaluation By increasing independence in evaluation, we can decrease certain types of bias including extreme conflict of interest Evaluation quality without independence does not assure credibility and does not protect learning 10 IAOD Evaluation Section Whz are independent evaluations necessary?

Assuring the Independence of Evaluations Organizational Independence The ES is part of IAOD and it performs its function independently from other WIPO Management functions to ensure impartial and independent reporting Behavioural Independence The evaluation reports are based on evidence and stakeholders are consulted at the various stages of the evaluation process. The budget for the evaluation is managed solely by IAOD Protection from outside interference IAOD Evaluation Section is be responsible for designing and executing the evaluation and evaluators results will not be subject to overruling or influence by any external authority Avoidance of conflict of interest IAOD will assure that the evaluators undertaking the evaluation will in no manner have an official, professional, personal or financial relation with any WIPO program and that he/she does not have any current or previous involvement with the development oriented activities or the entity being evaluated at a decision making level, or in a financial management or accounting role; or seeking employment with the Organization. 11 IAOD Evaluation Section

Are we doing the right things? And are we doing them right? To know if we are achieving an effect (+/-) on the intended beneficiaries To contribute to learning To support accountability and transparency to stakeholders specially Member States. To inform the CDIP, other stakeholders and management To inform the planning, programming, budgeting, implementation and reporting cycle To contribute to the improvement of the performance of on- going and future activities and to the enhancement of institutional and development effectiveness. Why do we need to evaluate the DA and DOA? (1) 12 IAOD Evaluation Section

To enhance the generation and use of value-added evaluative information for: Decision-making process concerning the improvement of present and future activities Policy formulation and review by Member States Management oversight by the Director General 13 IAOD Evaluation Section Why do we need to evaluate the DA and DOA? (2)

Establishing a program theory: how the intervention will benefit the beneficiaries and the potentials for side-effects Setting up a monitoring framework which includes expected results targets, baselines, SMART KPIs including systems and data base for data collection Allow comparison: before and after the intervention Plan the evaluation at the start of the intervention Define the evaluation approach, methodology and allocated a budget and time for it 14 IAOD Evaluation Section Requirements for evaluating the DA and DOA?

What will guide the evaluation of DA and DOA? The WIPO Evaluation Policy approved by the DG in 2007 guide the undertaking of the independent evaluations The Evaluation Policy lays a framework for the evaluation function of WIPO The Evaluation Policy provides a comprehensive framework for planning and conducting the evaluation following international best practice, taking due account of the Norms and Standards for Evaluation in the UN System adopted by the United Nations Evaluation Group (UNEG), and other international good practices. The Policy is applied to all independent evaluations 15 IAOD Evaluation Section

Relevance Are the Organization’s development oriented activities suited to the priorities and policies of the target group? Effectiveness Are the development oriented achieving its objectives? Efficiency Is the Organization for the implementation of its development oriented activities using the least costly resources possible in order to achieve the desired results? Impact What positive and negative changes are being produce by a development oriented activities, directly or indirectly, intended or unintended? Sustainability Are the benefits of the Development Agenda likely to continue after the Organization’s funding has been withdraw? Coverage which groups are included or excluded from the development oriented activities, and the differential impact on those included and excluded? Coherence Are other policies and programs affecting the implementation of the Development Agenda? Co-ordination How are the development oriented activities harmonizing efforts with WIPO programs, other aid agencies and alignment with country priorities and systems? DA and DOA Are we doing the right things? Are we doing them right? Where is the Evidence? 16 IAOD Evaluation Section What are the evaluation criteria applied to the evaluation of DA and DOA?

Evaluation will examine the processes and contextual factors to understand why, and to what extent, expected results have been achieved, and what impact they have had. Evaluation will recommend and suggest better ways of achieving expected results. Evaluation will identify good practices and lessons learned. What other information will the evaluation deliver? 17 IAOD Evaluation Section

Types of Evaluations

Independent Evaluations vs Costs Country Level Evaluations Program Evaluations Thematic Evaluations Systemic Evaluations Organizational Assessments Project Level Evaluations only Above Sfr 1 mio 19 IAOD Evaluation Section

Summative EvaluationsFormative Evaluations DA and DOA evaluation timing Development Intervention Starts/ about To start Development Intervention Is in the middle Of implementation Development Intervention is completed Time Stage of the Development Intervention Type of Evaluation Ex-ante evaluationMid-term evaluation Ex-post or final evaluation DA and DOA 20 IAOD Evaluation Section

Evaluation approach for the Development Agenda and Development Oriented Activities

Utilization-focused evaluation approach 1. ToRs drafting Participatory planning with Key stakeholders and partners 2. Evaluation process Supporting the evaluation team with a Resource or learning group 3. Draft findings/ Draft report Consultation with key stakeholders And workshops 4. Evaluation report Dissemination to a wider public 5. Follow-up day Assisting program managers with the Implementation of recommendations 22 IAOD Evaluation Section