1 Designing a Monitoring and Evaluation System for a Rural Travel and Transport Project Michael Bamberger Gender and Development Group The World Bank RTTP Conference Pretoria, South Africa December

2 A good M/E system helps ensure that a project Is being efficiently implemented Is reaching the intended target groups Is achieving its intended objectives

3 A good M/E system will  Monitor the use of project inputs  Monitor the effectiveness of the project implementation process  Monitor the production of project outputs  Assess project impacts on the target communities  Assess the effectiveness of project outputs in producing the intended short-term and long-term impacts.  Assess the extent to which these impacts can be attributed to the effects of the project.

4 An RTT M/E system should Extract lessons and best practices for the design of future projects Provide guidelines for the modification of project design Provide adequate data for the evaluation of program impact

5 The Basic Monitoring and Evaluation Model

6

7 Stages of the project cycle covered by the M/E system Project planning and inputs Implementation process Output Short and long term impacts Project sustainability

8 Project implementation and impacts are also affected by The organizational and institutional context The economic and political context The social and economic characteristics of the communities

9

10 Key questions for designing a M/E system What indicators should be used to measure and monitor each stage of the project? How should the information be collected? How to measure quality and effectiveness? How and when to communicate findings to managers and stakeholders?

11 Key questions continued…. How do we assess the benefits and impacts produced by the project? How to assess whether pilot projects have been successful.

12 Developing and using indicators Good indicators should Cover all stages of the project cycle Provide essential information needed to monitor progress, identify problems and assess results Provide information which can easily be used by managers and policy makers Be objective and easily measurable Include the minimum number possible

13 Examples of indicators for a feeder road project Planning and inputs Implementation process Outputs Stakeholder groups consulted Funds approved and available Equipment available All community included in planning and implementation Targets set for women representatives Local organizations established at district level Km. Roads constructed/ upgraded

14 Feeder road indicators page 2 ImpactsIncreased household income Higher proportion of girls attending school Increased agricultural production Wider range of goods available in villages

15 Feeder road indicators page 3 SustainabilityCash payments for road maintenance Community labor for road maintenance Condition of roads 1, 2 and 5 years after construction

16 M/E systems must identify all stakeholders and ensure their information needs are covered. All community groups affected by the project Local government agencies National government agencies NGOs International development agencies

17 Evaluation studies and evaluation systems

18 Individual M/E studies can be conducted each using the methods best suited to the particular issue being studied

But to assess the overall progress of a program involving many different activities …. An evaluation system is required

Logical framework analysis [LOGFRAME] Is one of the best methods for setting up a monitoring and evaluation system

21 LOGFRAME helps define the basic assumptions on which the project design and implementation systems are based.

22 LOGFRAME monitors the following logical sequence of activities……

23 If certain inputs are used effectively then certain outputs will be produced if the outputs are produced then certain impacts will be achieved

24 Example Using LOGFRAME to monitor and evaluate the impacts of a feeder road on women

25 If feeder roads are upgraded then women will take more agricultural produce to market then household income will increase and women’s economic status will improve

26 Main kinds of M/E studies See Handout Annex I for details Question 1.Are RTTP activities being implemented on schedule? 2.Are women fully involved in all stages? 3.Are the activities producing their intended benefits? Study Monitoring study Gender responsiveness assessment study Impact evaluation

27 Main kinds of studies page 2 Question 4.Are the activities sustainable? 5.Were the pilot interventions successful and could they be replicated? Study Sustainability assessment Replicability assessment