Strategic Planning for Learning Organizations Session 3. Impact and the Role of Monitoring, Evaluation and Impact Assessment of Agricultural Research for Development

Objectives of Session 3 Define impact and how it is related to strategic planning Define impact assessment (IA) Discuss timing and purpose of IA Discuss impact chain and strategic planning Discuss types of impact

Objectives of Session 3 (cont.) Define monitoring and evaluation (M&E) and their role Discuss the relationship between M&E Discuss different types of evaluation Discuss why a good plan is the basis for an M&E system

Definition of Impact people Means different things to different Direct product of a program (output) Effect of the direct product on ultimate users- “People-level” impact

Impact Special form of evaluation Deals with effects of (research) output on target beneficiaries Attempts to look at both intended and unintended effects Begins to occur when there is behavioral change among potential users or the system

Impact and Strategic Planning In strategic planning, the focus is largely on people-level impact Strategic planning is considered as a tool for achieving transformation and people-level impact Logical framework at the level of strategic planning spells out the outputs and indicators Achievement of the mission of the organization contributes to the system or sectoral people-level impact

Impact Chain Defined as the path from the lowest level of intervention (inputs) to achievement of people- level impact Strategic planning starts at the end of the impact chain for motivation of lower levels of plan Chain specifies all steps: Inputs and activities of a program, related projects & associated components Outcomes (Immediate, Intermediate and Ultimate) Direct and indirect effects Benefits to the ultimate beneficiaries — society

Impact Pathway Inputs from collaborating institutions Collaborative activities Outputs Immediate outcome Intermediate outcome Ultimate outcome (also referred to as people-level impact or developmental impact)

The impact pathway Agricultural research for development agencies Organizational sub-components, such as projects and group activities Agricultural research for development Organizational components, such as programs Individual agents Different agents working towards a shared higher level objective Responsibility for different products and services toward a higher level objective Responsibility for intermediate results (subproducts and services) Responsible for specialized inputs Responsibility for sub-intermediate results National System Level Organizational Program Project Activity Project Goal Projects Individuals Agricultural research for development agencies Organizational sub-components, such as projects and group activities Agricultural research for development Activ. Goal Activ. Purpose Activity Outputs Organizational components, such as programs Individual agents Different agents working towards a shared higher level objective Responsibility for different products and services toward a higher level objective Responsibility for intermediate results (subproducts and services) Prog Goal Org Goal System Goal Project Purpose Prog Purpose Org Purpose Project Outputs Prog Outputs Activities Programs Agencies System Outputs System Purpose Org Outputs Responsible for specialized inputs Responsibility for sub-intermediate results National System Level Organizational Program Project Activity Individuals Agricultural research for development agencies Organizational sub-components, such as projects and group activities Agricultural research for development Activ. Goal Activ. Purpose Activity Outputs Organizational components, such as programs Individual agents Different agents working towards a shared higher level objective Responsibility for different products and services toward a higher level objective Responsibility for intermediate results (subproducts and services) Prog Goal Org Goal System Goal Project Purpose Prog Purpose Org Purpose Project Outputs Prog Outputs Activities Programs Agencies System Outputs System Purpose Org Outputs Responsible for specialized inputs Responsibility for sub-intermediate results National System Level Organizational Program Project Activity Individuals Agricultural research for development agencies Organizational sub-components, such as projects and group activities Agricultural research for development Activ. Goal Activ. Purpose Activity Outputs Organizational components, such as programs Individual agents Different agents working towards a shared higher level objective Responsibility for different products and services toward a higher level objective Responsibility for intermediate results (subproducts and services) Prog Goal Org Goal System Goal Project Purpose Prog Purpose Org Purpose Project Outputs Prog Outputs Activities Programs Agencies System Outputs System Purpose Org Outputs Responsible for specialized inputs Responsibility for sub-intermediate results National System Level Organizational Program Project Activity Project Goal Projects Project Goal Projects System Goal System Purpose Org Goal System Outputs Org Purpose Prog Goal Agencies Org Outputs Prog Purpose Programs Prog Outputs Project Purpose Activ. Goal Project Outputs Activ. Purpose Activities Activity Outputs Individuals

} Types of Impact Institutional impact Environmental impact People- Production impact level Economic impact impact Socio-cultural impact

Impact Types, Techniques and Methods INTERMEDIATE IMPACT Institutional changes Changes in the enabling environment Survey, Monitoring Simple Comparison/ Trend Analysis DIRECT PRODUCT OF RESEARCH Effectiveness Analysis using Logical Framework Simple Comparison – Target vs. Actual ECONOMIC IMPACT Micro, Macro, Spillovers Econometric Approach Surplus approach Production Function Total Factor Productivity Index number methods and derivatives SOCIO-CULTURAL IMPACT Socioeconomic Survey Adoption Survey Comparison Over Time ENVIRONMENTAL IMPACT Environmental Impact Assessment Various Qualitative Quantitative

Monitoring and Evaluation

Monitoring Traditionally defined as a continuous assessment of implementation schedule and program inputs Focus has changed from monitoring implementation to tracking results Monitoring is an internal management tool Question: Is the program doing things right?

Types of Monitoring Implementation monitoring traditionally used for projects Focused on data collection on inputs, activities and immediate outputs Designed to provide information on administrative, implementation and management rather than broader development effectiveness issues Process monitoring Can be used to monitor achievement of program outputs/results

Types of Monitoring (2) Results monitoring Focus on baseline data, indicators for outcomes and change among stakeholders More qualitative and quantitative information on the progress towards outcomes; Done in conjunction with strategic partners; and Strong on institutional learning Progress monitoring

Evaluation Periodic assessment of the relevance, performance, efficiency, quality and impact of an organization, a program, project or activity in relation to the objectives and goals that should be achieved

Evaluation Any form of systematic assessment At any given stage in the program/project cycle Periodic assessment of relevance, performance, efficiency, quality, and impact of an organization, program, project or activity in relation to objectives and goals to be achieved Consists of comparisons, usually involving temporal and spatial information from inside and outside the project

Evaluation Aspects covered Performance Quality Relevance Efficiency Question: Is the program/project doing the right things?

Relationship of Monitoring and Evaluation Information from other Sources Information From Monitoring Monitoring Recording (data) Analysis Analysis Recommendations Reporting (information) Affirmation or Modification of Objectives, Resources & Processes Corrective Action at the Operational Level Storage