1. How to Prepare and Manage FFP Mid-term Evaluations Food and Nutrition Technical Assistance III Project (FANTA) FHI 360 1825 Connecticut Ave., NW Washington, DC 20009 Tel: 202-884-8000 Fax: 202-884-8432 Email: fantamail@fhi360.org Website: www.fantaproject.org HOW TO PREPARE AND MANAGE FFP MID-TERM EVALUATIONS Pamela Velez-Vega, Monitoring & Evaluation Research Associate September 22 nd, 2015

2. How to Prepare and Manage FFP Mid-term Evaluations 2

3. 3 Session Objectives Participants will: 1.Recognize requirements for a mid-term evaluation (MTE) 2.Identify steps to prepare and manage a MTE

4. How to Prepare and Manage FFP Mid-term Evaluations 4 Case Example Your organization was awarded a 5-year FFP development food assistance project in Madagascar. You are the Chief of Party.

5. How to Prepare and Manage FFP Mid-term Evaluations 5 Roles of USAID and Awardee Project Staff

6. How to Prepare and Manage FFP Mid-term Evaluations 6

7. 7 MTE Objectives MTE design and support should be strongly oriented toward Objective 1 1 a)Adherence to terms agreed b)Target communities’ perceptions of intervention c)Factors challenging or supporting implementation 2 Intended and unintended changes 3 Adjustments to implementation and ToC or RF Photo by: Alex Harsha, Medic Mobile

8. How to Prepare and Manage FFP Mid-term Evaluations 8 Key Evaluation Questions Focus of evaluation a)Adherence to terms b)Target perceptions c)Factors 3. Adjustments to implementation 2. Intended and unintended changes 1.

9. How to Prepare and Manage FFP Mid-term Evaluations 9 MTE Methods FFP discourages large-scale quantitative surveys

10. How to Prepare and Manage FFP Mid-term Evaluations 10 MTE Methods Unstructured or semi-structured interviews Observation Mostly Qualitative

11. How to Prepare and Manage FFP Mid-term Evaluations 11 MTE Methods Project data and secondary data analysis Small-scale surveys: Test hypothesis Capture perceptions Cross-check What kinds of quantitative data collection methods are encouraged?

12. How to Prepare and Manage FFP Mid-term Evaluations 12 MTE Methods: Sampling Site selection Informant selection Project intervention

13. How to Prepare and Manage FFP Mid-term Evaluations 13 Cooperative and iterative process among Awardee FFP AOR and M&E team and Mission FFP AOR and M&E team and Mission Preparing the Statements of Work (SOW)

14. How to Prepare and Manage FFP Mid-term Evaluations 14 Illustrative MTE SOW A.Introduction (overview, project background) B.MTE Objectives C.MTE Methods D.Contractor Responsibilities E.MTE Team Composition, Qualifications and Roles F.Project Responsibilities G.Intellectual Property H.Ethical Guidelines

15. How to Prepare and Manage FFP Mid-term Evaluations 15 Project Responsibilities 1.Prepare SOW 2.Select evaluation team 3.Provide secondary data 4.Serve as informant 5.Provide logistical advice 6.Offer administrative support (optional)

16. How to Prepare and Manage FFP Mid-term Evaluations 16 Project Responsibilities: Provision of secondary data Geographic orientation documents Proposal and amendments Reports to USAID M&E plan Intervention protocol descriptions Intervention reports and data Exit strategy and sustainability plan

17. How to Prepare and Manage FFP Mid-term Evaluations 17 Illustrative MTE Statement of Work (SOW) A.Introduction (overview, project background) B.MTE Objectives C.MTE Methods D.Contractor Responsibilities E.MTE Team Composition, Qualifications and Roles F.Project Responsibilities G.Intellectual Property H.Ethical Guidelines

18. How to Prepare and Manage FFP Mid-term Evaluations 18 Conclusions Evidence Recommendations Interpretation of the data Data Proposed actions for management Contractor Responsibilities: MTE Report

19. How to Prepare and Manage FFP Mid-term Evaluations 19 Illustrative MTE Statement of Work (SOW) A.Introduction (overview, project background) B.MTE Objectives C.MTE Methods D.Contractor Responsibilities E.MTE Team Composition, Qualifications and Roles F.Project Responsibilities G.Intellectual Property H.Ethical Guidelines

20. How to Prepare and Manage FFP Mid-term Evaluations 20 Team Composition External evaluator quantitative and qualitative All technical sector and cross-cutting themes EWS, Gender, Environment Ag & Livelihoods MCHN Unbiased

21. How to Prepare and Manage FFP Mid-term Evaluations End of Year 1 End of Year 2 End of Year 3 MTE timeline 21 Project award SOW draft to USAID within 15 months of award MTE submission Approval and Procurement of evaluators Secondary data review and work plan prep Data collection and field validation Data analysis and report drafting USAID feedback and report finalization 1 1 2 2 3 3 Set up MTE timing within first year

22. How to Prepare and Manage FFP Mid-term Evaluations End of Year 1 End of Year 2 End of Year 3 MTE timeline 22 Project award SOW draft to USAID within 15 months of award MTE submission Approval and procurement of evaluators Secondary data review and work plan prep Data collection and field validation Data analysis and report drafting USAID feedback and report finalization 1 1 2 2 3 3 Set up MTE timing within first year (3-4 months) (3-4 months for agreement & final approval) (4-6 weeks) (6-8 weeks excluding travel time) (6-8 weeks) (4-8 weeks)

23. How to Prepare and Manage FFP Mid-term Evaluations 23 MTE Budget Minimum $150,000-$200,000 per award Increased coverage of interventions in the field Travel time to and among intervention sites

24. How to Prepare and Manage FFP Mid-term Evaluations 24 Common Pitfalls to avoid! Late or incomplete provision of secondary data Team leader with insufficient evaluation training Logistics bias site selection

25. How to Prepare and Manage FFP Mid-term Evaluations 25 Common Pitfalls to avoid (cont.) ! MTE team not aware of logistical responsibilities (e.g. vehicles, interpreters) MTE report: Recommendations not supported by conclusions No evidence provided to support conclusion

26. How to Prepare and Manage FFP Mid-term Evaluations 26 Session Objectives (Recap) Participants will: 1.Recognize requirements for a mid-term evaluation (MTE) 2.Identify steps to prepare and manage a MTE

27. How to Prepare and Manage FFP Mid-term Evaluations 27 Resources Draft: USAID’s Office of Food for Peace Policy and Guidance for Monitoring, Evaluation, and Reporting for Development Food Assistance Projects Submit inputs to FACG@amexdc2.com by October 29, 2015!

28. How to Prepare and Manage FFP Mid-term Evaluations This presentation is made possible by the generous support of the American people through the support of the Office of Health, Infectious Diseases and Nutrition, Bureau for Global Health, U.S. Agency for International Development (USAID) and the Office of Food for Peace, Bureau for Democracy, Conflict and Humanitarian Assistance, under terms of Cooperative Agreement No. AID-OAA-A- 12-00005, through the Food and Nutrition Technical Assistance III Project (FANTA), managed by FHI 360. The contents are the responsibility of FHI 360 and do not necessarily reflect the views of USAID or the United States Government. 28