iSTEP: Technology Field Research in Developing Communities Instructor: M. Bernardine Dias CAs: Sarah Belousov and Ermine Teves Spring 2009

Lecture Monitoring and Evaluation

3 Outline M&E IRB Q&A

Monitoring and Evaluation

5 The Need Project managers, donors, and other stakeholders need to know the extent to which their projects are meeting their objectives and leading to their desired effects. M&E build greater transparency and accountability in terms of use of project resources. Information generated through M&E provide project staff with a clearer basis for decision-making. Future project planning and development is improved when guided by lessons learned from project experience.

6 Relationship Between M&E

7 Logical Framework Approach

8 Methods and Tools

9 Methods and Tools

10 Methods and Tools

11 Methods and Tools

12 Methods and Tools

13 Methods and Tools

14 Selecting M&E Methods What information is needed? Of this information, how much can be collected and analyzed in a low- cost and practical manner? How accurate will the information be? Will the methods get all of the needed information? What additional methods should and could be used if additional information is needed? Will the information appear as credible to decision makers? Are the methods appropriate for the target group? If group members are illiterate, the use of questionnaires might not be appropriate unless completed by the evaluators themselves. Who can administer the methods? Is training required? How can the information be analyzed?

15 Types and Models M any types of evaluations – but 2 main categories: –Formative (process evaluation) Examines the development of the project and may lead to changes in the way the project is structured and carried out –Summative (outcome or impact evaluation) Examines what a project has actually accomplished in terms of its goals Popular evaluation methods –Pre-test Post-test model Measures the situation before the project starts and repeat the same measures after the project is completed. The differences or changes are attributed to the project –Comparison Group model Compares project results on two comparable groups at the same period of time, where one group represents beneficiaries of the project and the other represents a group that has not benefited from the project. The difference between the two groups are attributed to the project Key Evaluation Indicators are: Effectiveness, Efficiency, Relevance, Impact, and Sustainability

16 Planning a Monitoring System 1.Decide what should be monitored The careful selection of monitoring indicators organizes and focuses the data collection process 2.Decide how to gather information Select methods to track indicators and report on progress (observation, interviews, stakeholder meetings, routine reporting, field visits, etc.) 3.Decide who will gather information and when/how often Project staff at various levels will do most data collection, analysis and reporting. Staff should agree on what the monitoring report should include. 4.Decide on feedback process Progress reports should be reviewed by project staff and major stakeholders. Feedback should be collected by project managers on a regular basis. 5.Allocate resources for monitoring plan Needed funds and staff time should be allocated to ensure effective implementation

17 Planning an Evaluation 1.What are the purposes of the evaluation? Identify a manageable number of evaluation purposes and prioritize them. Ask who needs what type of information and for what reason. 2.What evaluation model is the most appropriate for the project? The evaluation model that a specific project would utilize should be selected during the project design phase. 3.What is the timing of evaluation within the project cycle? Evaluation timing is determined by the project plan, the identification of significant problems during monitoring, donors’ request, etc. 4.What is the scope and focus of the evaluation? Identify the geographic area, type of activity and time period that the evaluation should cover to clarify the types of questions to be asked. 5.What methods of gathering data will be used? Existing data should be identified and assessed. Note that extensive data gathering is time-consuming, expensive, and can result in a lot of unnecessary information. 6.What resources are needed for the evaluation? Sufficient human, financial and logistic resources should be allocated.

18 Analysis and Dissemination Data management is very important and needs to be planned so that effective analysis and dissemination can follow Both quantitative and qualitative data can be useful/required Numerical and statistical methods are often used to analyze quantitative data It is important to find effective ways to combine the quantitative and qualitative data analysis to produce overall results and recommendations Effectively disseminating evaluations (through presentations and reports) is important for many reasons Suggested content for evaluation report: