1. Monitoring and Evaluation of GeSCI’s Activities GeSCI Team Meeting 5-6 Dec 2007

2. Why should GeSCI Monitor and Evaluate our activities? To Evaluate: The quality and effectiveness of our advice To Promote: to MoE, partners, stakeholders, donors To Acknowledge: Important Milestones To Learn: GeSCI as a “learning organisation” TO IMPROVE

3. The challenges to GeSCI Monitoring and Evaluating our activities Without critical analysis of the M&E information, it will not turn into knowledge Without “learning agility” M&E will just increase everyone’s workload M&E will not be dynamic if it doesn’t feed back into the learning cycle “Hidden assumptions” – do we all think it is important enough?

4. Questions to ask at the beginning of programme 1. Are there clear programme objectives? 2.Are there aligned benefits or performance indicators defined for the project? 3.Are those benefits measurable? 4.Do the benefits have owners / beneficiaries? 5.Is there a baseline against which to measure each benefit? 6.Is there clear accountability for each objective? 7.Have the objectives, benefits and targets been communicated? 8.Is the process / responsibility for measuring the benefits established?

5. Questions to ask during the programme 1.Are the benefits being monitored? 2.Are remedial actions in place? 3.Is the progress being communicated? 4.Are all of the programme activities aligned to the delivery of the benefits? 5.How engaged are the benefits owners/recipients of change? 6.Is the senior behaviour supportive of the benefits delivery? 7.Are the benefits aligned to the objectives of individuals?

6. What is Outcome Mapping? Research carried out by IDRC’s Evaluation Unit and Michael Quinn Patton Takes the form of periodic facilitated workshops and follow up interviews with beneficiaries, partners and stakeholders Outcome Mapping assesses contributions made by development projects and programmes, or organizations to the achievement of outcomes Outcomes are defined as changes in the behaviour, relationships, activities, or actions of the programme’s partners Outcome Mapping should not replace traditional forms of M&E that look at changes in state. - OM to support our workplans?

7. Why not focus on impact? Demonstrating impact requires isolating key factors that cause results and attributing them to one program or organisation Linear “cause and effect” thinking contradicts the complex process of development Feedback on performance should concentrate on improving rather than proving, understanding rather than reporting, creating knowledge rather than taking credit

8. Outputs of Workshop Stage 1: Design Stage Planning: Why? How? Who? vision, mission, boundary partners What? outcome challenges, progress markers Stage 2: Performance Monitoring Strategy maps Organisational Practices Journal A monitoring framework outcome journal, strategy journal, performance journal Stage 3: Evaluation Evaluation Plan

9. The Questions that Outcome Mapping will help to answer : In designing and articulating the program’s logic - What are our development goals? - How can our programme contribute to those development goals? - How can we help our partners contribute to the broader development goals? Recording internal and external monitoring data - How far have our beneficiaries progressed towards achieving outcomes? - What are we doing to support the achievement of those outcomes? - How well have we performed? Indicating cases of positive performance and areas of improvement - What worked well? Why? - How can we maximise our contributions?

10. The Questions that Outcome Mapping will help to answer (continued) : Evaluating intended or unexpected results - Who changed / what changed? How? - If they did not change as expected, do we need to do something different or reorient our expectations? Gathering data on the contribution that a program made to bringing about changes in its partners - What activities / strategies were used? - How did the activities influence the individuals, groups, or institutions to change?

11. Examples of Outcome Mapping SchoolNet Namibia –Used in 2005 to explore details of the programme and lay down a road map for replication in other situations Uganda Health Information Network - Used in 2004 to monitor their organisational practices, progress made towards change, strategies employed to achieve outcomes

12. School Net Namibia Example

13. Boundary partnerOutcome challenge Government officials and policymakers Education Department through the Ministries of Higher Education and Basic Education. Endorsement of the ICT initiatives, approval of the implementation details and establishing guidelines for future implementation.

14. School Net Namibia Example Boundary partnerOutcome challenge Ministries of Education(s) Endorsement of the ICT initiatives, approval of the implementation details and establishing guidelines for future implementation. Progress markers Expect to see 1.Negotiating with SchoolNet and other service providers on details of implementation 2.Setting a priority list for implementation Like to see 1.Efficient coordination of partnerships for delivery of ICT services to schools. 2.Establishing guidelines for integrating ICT effectively into learning and teaching Love to see 1.Creating the critical mass of ICT skills at school level to be able to realise the country’s vision for the future. 2.Setting specific guidelines on cost-effectiveness for ICT solutions for schools or putting implementation out to tender to see who offers the best option

15. School Net Namibia Example Organisational Practices 1.Seeking feedback from key informants A feedback mechanism was put in place from the outset and maintained throughout the life cycle of the programme through ongoing discussions with the key stakeholders

16. Other methods of M&E Logical Framework Analysis –Work Plans and Monitoring and Evaluation method combined –A framework incorporating the concepts of continued rationale, efficiency, effectiveness, effects and lessons learned, which is used during implementation/completion and after the completion stages for ex- post evaluations Results Based Management –Feedback on actual outcomes and goals of programme actions –Help answering to questions: what are the goals; are they being achieved; and how can achievement be proven? Balanced Scorecard –A management system to enables organizations to clarify their vision and strategy and translate them into action –A feedback around both the internal processes and external outcomes in order to continuously improve strategic performance and results –Provides a framework for managing the implementation of strategy while also allowing the strategy itself to evolve in response to changes in the organization’s competitive market, and technological environments.

17. If you do not measure results, you cannot tell success from failure. If you cannot see success, you cannot reward it. If you cannot reward success, you are probably rewarding failure. If you cannot see success, you cannot learn from it. If you cannot recognize failure, you cannot correct it. If you can demonstrate results, you can win public support. Adapted from Osborne & Gaebler 1992 THANKS