Monitoring and Evaluation What are we talking about? 28/03/2017 Monitoring and Evaluation What are we talking about? Module 5 Session 8

Monitoring and Evaluation 28/03/2017 Monitoring and Evaluation 1. What is it? 2. Why do it? 3. Who does it? 4. When do we do it? 5. How do we do it? 6. What does this mean for me?

28/03/2017 What is it? Monitoring Management’s continuous examination of progress achieved during the implementation of an undertaking to track compliance with the plan and to take necessary decisions to improve performance. Key elements: - continuous examination of implementation progress - tracking compliance against planned objectives - generating data and information on performance to enable corrective measures to be taken

28/03/2017 1. What is it? Evaluation Assessment, as systematic and impartial as possible, of an activity, project, programme, strategy, policy, topic, theme, sector, operational area, institutional performance, etc. It focuses on expected and achieved accomplishments, examining the results chain, processes, contextual factors and causality, in order to understand achievements or the lack thereof. It aims at determining the relevance, impact, effectiveness, efficiency and sustainability of the interventions and contributions of the organizations

1. What is it? Evaluation (cont.) 28/03/2017 1. What is it? Evaluation (cont.) Key elements: assessment conducted at a single point in time (before, during or after). focuses on determining whether what was planned actually happened, and why it did or did not happen. Assessing: relevance – whether the intervention was appropriate impact – whether it made a difference in the lives of people effective – whether it achieved what it set out to efficient – whether it did so at the lowest cost sustainable –whether it will leading to lasting change.

Other similar functions that are not monitoring and evaluation 28/03/2017 Other similar functions that are not monitoring and evaluation Inspection: a general examination that seeks to identify vulnerable areas and malfunctions and to propose corrective action. Investigation: a specific examination of a claim of wrongdoing and provision of evidence for eventual prosecution or disciplinary measures.

Other similar functions that are not monitoring and evaluation 28/03/2017 Other similar functions that are not monitoring and evaluation Audit: an assessment of the adequacy of management controls to ensure the economical and efficient use of resources; the safeguarding of assets; the reliability of financial and other information; the compliance with regulations, rules and established policies; the effectiveness of risk management; and the adequacy of organizational structures, systems and processes. Research: a systematic examination designed to develop or contribute to knowledge.

Different oversight functions 28/03/2017 Different oversight functions Programme Cycle Management Monitoring Audit Evaluation Inspection Continuous Assessment Research Performance Compliance Modelling Wrong Doing

Where M&E fits in the cycles 28/03/2017 Where M&E fits in the cycles Policy Strategy Program Project Monitoring Planning Evaluating Programming Implementing

28/03/2017 2. Why do it? Why monitor? What gets monitored is more likely to get done. If you don’t monitor performance, you can’t tell success from failure. If you can’t see success, you can’t reward it. If you can’t recognise failure, you can’t correct it. If you can’t demonstrate results, you can’t sustain support for your actions.

28/03/2017 Why evaluate? Understand why and the extent to which intended and unintended results are achieved, and their impact on stakeholders Important source of evidence on the achievement of results and institutional performance, thus is one basis for corporate accountability Important contributor to building knowledge and organizational learning.

28/03/2017 What’s the difference? They are different, but interrelated functions, as they both contribute knowledge as a basis for accountability and enhanced performance. Monitoring is an internal, repetitive, operations and management function. Evaluation is often external, periodic/ snapshot, in greater depth and asking different questions. Monitoring asks the question “Are we doing things right”? Evaluation asks “Are we doing the right things?” and “Are there better ways of achieving the results?”

Development project example.. 28/03/2017 Development project example.. Problem: Low access to and use of Information and Communications Technology (ICT) by civil servants, youth and women in rural Uganda Diagnose: Government study to identify why there is low access and use of ICT Response: Government sets up a 2 year project with the objective to improve the use of ICT among these groups. Project aims (or outcomes): public access to ICT in Uganda improved; basic computer knowledge increased Project actions and deliverables: establish 50 telekiosks across the country, rehabilitate 160 ICT training centres; train Ministry of ICT staff; provide technical assistance in policy formulation.

Development project example…continued… 28/03/2017 Development project example…continued… Monitor: Inputs - money spent, computers brought, consultants hired Activities and Outputs (actions) number of telekiosks operational; number of ICT training centres open to public; number of training sessions held; number of registered domain names. Aims (outcomes) number of hours per month telekiosks used; number of graduates from ICT training centres; number of registered ISPs.

Development project example…continued… 28/03/2017 Development project example…continued… Evaluate: Was the problem correctly identified? Was the response (project) correctly designed to address it? Has it worked- has access to ICTs improved in these target groups? If not, why not? What could be done better in a future project of this type? Follow-up: Adapt existing project to respond to evaluation findings/ close down project/ start new project

Discussion space (10 mins) 28/03/2017 Discussion space (10 mins) Discuss the example of the ICT project Does the example make sense to you? Do you monitor and evaluate your projects? How do you do it?

So.. 3. Who does M&E? Monitoring 28/03/2017 So.. 3. Who does M&E? Monitoring Typically done by the subject, i.e. self-monitoring. Evaluation Typically done by external party, to ensure the integrity of the findings (i.e. remove bias), and provide insight (expertise) But….depends on the objectives… Self-Evaluation – done by the subject, when the primary objective of the evaluation is to learn about the strengths and weaknesses of what is being evaluated. External Monitoring – where there is a need for very exact standards, or there is a lack of trust- external parties may monitor the actions. E.G. elections

4. When is it done and who decides? 28/03/2017 4. When is it done and who decides? Monitoring is a continuous activity, reports on monitoring are typically produced quarterly. Evaluation is done at different phases: Before an initiative is planned. This is called a pre-appraisal or baseline During an initiative. Typically mid-way through, to check how it is going, whether it is still the right thing to be doing. At the close of an initiative. To determine whether the intended actions have been completed, and what different it has made. Some time after the initiative. Change is often slow, so it may be necessary to evaluate some years after to look at the effects. E.G. education- build a school, train teachers, provide meals. Aim=increased enrollment of kids- this may happen 6 months, 1 year or more down the line.

4. When is it done and who decides? 28/03/2017 4. When is it done and who decides? Evaluation is also done to address wider concerns: Where actions in a sector or area have been present for a long time, e.g. Evaluation of the PEAP 1997-2007 Where those involved, managing or governing have a particular concern, e.g. Evaluation of National Agricultural Advisory Development Service (NAADS) Where those involved or managing are interested to learn more, e.g. Evaluation of Budget Support

But Evaluation does not always ask the right questions……. 28/03/2017 But Evaluation does not always ask the right questions……. Or tell the subject what it wants to hear! 'We did a full work-up - heart, lungs, bank account, credit...you can afford to live another 19 years.'

5. How is evaluation done? Who is involved and when? 28/03/2017 5. How is evaluation done? Who is involved and when? Depends on the intentions. Participatory evaluation is with full involvement of users at all phases- design, implementation, write-up, follow-up Why not always use a fully participatory approach? Time, Independence, Intention.

5. How is evaluation done? What methods are used? 28/03/2017 5. How is evaluation done? What methods are used? Desk review of relevant documents (project documents, annual reviews, donor-specific, etc) Key informant interviews: with key partners and information stakeholders both at central and field levels. Drawing on specific check-listed questions Focus group discussions: internally and external parties both at central and field levels. Gaining consensus on key issues. Sample surveys: of effects and impacts of initiatives as and where necessary

Discussion space (30 mins) 28/03/2017 Discussion space (30 mins) 6. What does this mean for me? Break in to groups of 4 or 5 persons Share your own examples of monitoring and evaluation. Identify and discuss any good practices and challenges to conducting m&e in your district. Report back in plenary