1 of 19 Evaluating an Information Project Defining Content © FAO 2005 IMARK Investing in Information for Development Evaluating an Information Project Defining Content

2 of 19 Evaluating an Information Project Defining Content Learning Objectives At the end of this lesson you should be able to: use a Logic Model to identify the if-then causal relationships in your project; understand how this Logic Model can be used to decide what you want to measure.

3 of 19 Evaluating an Information Project Defining Content Introduction In order to examine the issues involved in the planning of an evaluation, we are following a scenario: Dr Kumar, the Director of publications at SMAU University, must evaluate a newsletter. Dr Kumar and his colleagues have held the first Evaluation Management Committee meeting, to reach an agreement on the focus of the evaluation. The EMC conducted a preliminary analysis of the RICE NEWS project, examining Needs, Beneficiaries, Activities, Outcomes. Also, a SWOT analysis identified potential risks and benefits of the evaluation. The next step will be to define what to measure, and how.

4 of 19 Evaluating an Information Project Defining Content Deciding what to measure During the EMC meeting, we agreed that an impact of RICE NEWS is more farmer knowledge about fertilizer applications. How can we measure that? Should we ask how many farmers have read RICE NEWS? Or how many farmers actually knew more after reading it?

5 of 19 Evaluating an Information Project Defining Content Deciding what to measure It is important to decide first what to measure, and then how. Therefore: 1. agree on what issues to cover in the evaluation (e.g. farmer knowledge) 2. agree on what data to collect and how to do it....we can use something called a Logic Model..

6 of 19 Evaluating an Information Project Defining Content What is a Logic Model A Logic Model is a tool that helps understand the if-then relationships in our project. We can use it to understand what leads to what...

7 of 19 Evaluating an Information Project Defining Content What is a Logic Model What inputs in the project have led to what activities? What activities have led to what outputs? What outputs have led to what impacts (or outcomes)?

8 of 19 Evaluating an Information Project Defining Content The Usefulness of a Logic Model These are some of the questions that a Logic Model can answer: Are we clear as what the project was supposed to achieve? What do we really want to measure? Are we focusing on the right things? Are we likely to be able to collect good data about the issues we want to include in the evaluation?

9 of 19 Evaluating an Information Project Defining Content Analyzing the if-then relationships in the project To build a Logic Model, start by preparing some worksheets: the first one deals with the relationship between inputs and activities; the second with activities and outputs; and the third with outputs and outcomes.

10 of 19 Evaluating an Information Project Defining Content Analyzing the if-then relationships in the project Lets take the first worksheet: inputs and activities. What resources have been used for RICE NEWS? What activity has each input contributed to?

11 of 19 Evaluating an Information Project Defining Content Creating your own logic model Lets think about our resources, or inputs, one at the time. Lets start with funds coming from the Ministry of Agriculture......we used that money to buy some equipment for the production of RICE NEWS

12 of 19 Evaluating an Information Project Defining Content Creating your own logic model This is the first table: inputs and activities. SOURCETYPE OF INPUTSCONTRIBUTION TO WHICH ACTIVITIES? Ministry of AgricultureFundsPurchase of publishing equipment Mikuni State Agricultural University Publication professionals The Newsletter production process Mikuni State Agricultural University ScientistsResearch The extension department Extension agentsThe Newsletter distribution

13 of 19 Evaluating an Information Project Defining Content Creating your own logic model This first table is useful, to: evaluate if the RICE NEWS project has been organized efficiently (a performance evaluation); focus more on impact, and see how inputs and activities have contributed to outcomes. It will be necessary to use this first table, as well as the next two.

14 of 19 Evaluating an Information Project Defining Content Creating your own logic model Now the second table. For each activity noted in the first table, lets ask: What has been the main output? Who have been the main beneficiaries of that output?

15 of 19 Evaluating an Information Project Defining Content Creating your own logic model Here's the second table: activities and outputs. ACTIVITIESOUTPUTSBENEFICIARIES Research.Articles submitted to RICE NEWS. Articles accepted. The University Scientists The Newsletter production process. Number of trained staff. Participating students. The Publication Department. Students. Commercial Firms. Purchase of publishing equipment. Number of PCs, printers, presses etc. The University. The Publication Department. The Newsletter distribution. Number of copies distributed. Extension agents. Farmers. Agribusiness.

16 of 19 Evaluating an Information Project Defining Content Creating your own logic model And finally, the third table. Outputs and Intended Impacts (or outcomes): OUTPUTSINTENDED IMPACTS Articles submitted to RICE NEWS. Articles accepted. More relevant research. More scientists promotions. Number of trained staff. Participating students. More farmer knowledge of new developments in rice production practices. Number of PCs, printers, presses etc. More cost-efficiency in the production of RICE NEWS Number of copies distributed Increased farmer commitment to put knowledge into practice.

17 of 19 Evaluating an Information Project Defining Content Creating your own logic model A Logic Model for planning can help separate activities, from outputs, from impacts (or outcomes). By doing so, you can decide whether the main focus of your evaluation will be on performance, or impact, or both.

18 of 19 Evaluating an Information Project Defining Content Creating your own logic model You have your Logic Model. You are ready to use this Logic Model as a basis for formulating your detailed evaluation questions.

19 of 19 Evaluating an Information Project Defining Content Summary To agree on the detailed content of an evaluation, it is useful to build a Logic Model. A Logic Model is a tool that helps understand the if-then relationships in a project. A Logic Model can therefore help focus on what you really want to measure. The major benefit of using a Logic Model for planning your evaluation is to separate activities, from outputs, from impacts (or outcomes). It can also help evaluate if the objectives of each of these areas have been met.