1. SJI Impact Assessment 2014 Based on Impact Assessment for Development Agencies – Learning to Value Change (Chris Roche, Oxfam, digital file 2010)

2. Impact Assessment Sustained changes in people’s lives brought about by a particular intervention

3. Attribution “One of the critical questions that impact assessment traditionally sets out to answer is what caused any identified change. For most projects this means trying to determine if the changes that have occurred would have happened anyway, and the degree to which an observed change can be attributed to a given project or programme. This is notoriously difficult.”

4. Purpose To demonstrate past impact and to improve future practice

5. Objectives and Indicators Impact is the indicator to overall objective Outcome is the indicator to specific objectives In SJI’s case indicators are for the most part not defined (some organic agriculture and GNH indicators, CSOFF) -> Impact assessment exercise will help defining those indicators (participatory) for future planning/programming/project design

6. Objectives and Indicators For example: Specific objective: - Minimization of waste -> To what extent has it been achieved? Outcome indicator(s): - What are the outcomes that contributed to this change? - What outcomes can be attributed to the project? - What outcomes can be attributed to external factors? - Any negative/unintended (postive or negative) impacts of this change?

7. Impact Assessment Team SJI team! Internal training/capacity development exercise (no external evaluators/facilitators) External support from Dr. Gonsalves and Dr. Marchetti in design of impact assessment Other capacity development required?

8. Units/Levels of Assessment Individual Household Group/CBO Community/village Local CSO Institution

9. Participants Men, women, youth, rich, poor, beneficiaries, non- beneficiaries, project staff, donors, local government…

10. Research Tools Surveys Interviews Workshops/discussions Direct observation Participatory research Case studies/stories

11. Research Tools -> Importance of using several research tools and participants for cross- checking (triangulation)!

12. Research Method Quantitative method Specific objective: - Minimization of waste -> To what extent has it been achieved?  Ranking on scale between 1-10 before and after programme/project implementation (with or without project)  Will at the same time generate baseline info

13. Research Method Qualitative method Outcome indicator(s): - What are the outcomes that contributed to this change? - What outcomes can be attributed to the project? - What outcomes can be attributed to external factors? - Any negative/unintended (postive or negative) impacts of this change?  Semi-structured questions

14. Research Method Qualitative method Other questions: - Suggestions for improvement of the programme/project (relevance, efficiency, effectiveness, impact, sustainability) - Potential for replication, upscaling to other parts of the district/beyond - Potential for policy uptake/impact (integration into local government policy)  Semi-structured questions

15. Research Method Relevance: Real problems, needs, priorities of target group and beneficiaries, contextual appropriateness (policies) Efficiency:Benefits compared to resources spent Effectiveness:Results contribution to achievement of specific objectives Impact: Projects contribution to overall objective Sustainability:Benefits of project persisting after donor support, institutional and financial sustainability

16. Research Tools “Methods should evolve and be adapted based on a mutual analysis of their strengths and weaknesses; and one should monitor the extent to which the information gathered actually has an impact - for instance, whether it leads to change in the policies or practice of the project or organisation being assessed and whether it is meaningful to all parties.”

17. Communication Tools Community feedback Presentations and workshops with staff Multistakeholder workshops Report Case studies/stories Radio Video Etc.

18. Task of the Day For each SJI programme/project:  Determine units/levels of assessment  Determine participants  Determine research tools  Determine timelines

19. SJI Internal Impact Assessment Multiple impact assessments: i) SJI internal, ii) SJI partners/beneficiaries, iii) external evaluator, iv) donors Sometime next week Purpose: To conduct internal impact assessment and test out research tools