1. Harvesting outcomes from a global network dedicated to improving the life of vulnerable children worldwide
Goele Scheers
EES-2016 Conference Maastricht
Outcome Harvesting- Principles in Practice
Thursday 29 September 2016
15:45 – 15:15 PM
Room 0.11

2. Family for Every Child is a global alliance of national civil society organisations working together to mobilise knowledge, skills and resources to build a world where every child grows up in a permanent, safe and caring family, and to provide quality alternative care where needed

3. Outcome Harvesting for Family for Every Child
OH Evaluation: outcomes
Integrate OH into the M&E system of the network

4. 6 OH steps applied to Family for Every Child
1. Design the Harvest
2. Review documentation and draft outcomes
3. Engage network members, board and staff
5. Substantiation
6. Support the use of findings
4. Analyse and interpret

5. Principle 1: Ensure usefulness throughout the evaluation
1. Design the Harvest

6. Users and uses as defined during design
The EveryChild Board of trustees will use the findings to decide on a full transfer of its assets and resources to Family for Every Child and thereby complete the transition process.
Family for Every Child members, Board and Senior Management Team will use the findings as input for the Strategic Plan.

7. Useful evaluation questions
What are the emerging external outcomes telling us about the added value of the network, in particular in relation to Family’s conceptual framework goals?
What are the emerging internal and external outcomes telling us about the vibrancy and sustainability of the network, in particular in relation to Family’s strategic objectives?
What does this mean for the asset transfer decision and the strategic plan?

8. During the harvest
Deadline for asset transfer process was moved forward, design had to be adapted
After the harvest had been done: decision together with users not to substantiate. Outcomes deemed to be credible enough for internal use.
Users and evaluator decided not to do additional harvesting, because the results were needed quicker than anticipated to inform the strategic planning process

9. 2. Review documentation and draft outcomes
Principle 2: Harvest social change outcomes Principle 3: Formulate an outcome as an observable change Principle 4: Establish plausible influence or contribution, not control or attribution
2. Review documentation and draft outcomes

10. Document Review
Due to the strong focus on activity and output reporting, there were only a small number of outcomes generated from documents

11. 2 types of network outcomes
Internal outcomes: changes in the behaviour, relationships, or actions of the Family for Every Child members and bodies which strengthen and develop their collective capacity to achieve the network’s purpose.
External outcomes: changes in the behaviour, relationships, actions, policies or practices of individuals, groups or organisations outside of the Family for Every Child network that represent significant contributions towards the network’s purpose.

12. Example of an external outcome from Family
Outcome description: In June 2015, The Ministry of Women and Child Development of India invited Butterflies (Family’s Indian member) to lead the Foster Care Committee in developing guidelines, training modules and monitoring tools on foster care and included their contribution in the Foster Care guidelines that were launched in October Significance: This was a key moment in the development of state policy and guidance in India, when not only international NGOs and experts, but also national NGOs, including Butterflies were asked to contribute. Previously, there had been piecemeal and patchy development of foster care in India, led by internationally supported NGOs, but not strongly or consistently coordinated by the government. Contribution: The exposure of Butterflies to foster care through its membership of Family enabled it to respond to a request it received from its government, and thereby make a significant contribution to the government’s development of policy and guidance for foster care providers. Butterflies was able to respond within tight deadlines because of the help of network members finding and sharing with them the policy and practice examples from various countries.
Network contribution can be secretariat as well as members, board, etc
Pitfalls: wanting to formulate ‘more aware’ as an outcome

13. Example of an internal outcome from Family
Outcome Description: At the 2015 Assembly of Members in South-Africa, network members were leading the sessions. Significance: This outcome demonstrates that there has been a shift in power, decision-making and representation away from the secretariat to the members. It is an indication, that Family for Every Child is becoming a member-led network. Contribution: The secretariat, together with the board, have put the necessary structures in place that enabled network members to lead sessions. For example: the approach to having co-chairs for the Multi Country Initiatives meant that there were members who had been involved in the decision-making on the development and delivery of the projects. They were therefore informed enough and participating in the leadership of the projects enough to be able to lead sessions.

14. 3. Engage network members, board and staff
Principle 5: Facilitate the identification and formulation of outcomes
Principle 6: Nurture appropriate participation
3. Engage network members, board and staff

15. Outcomes were gathered during focus groups with secretariat staff, board and members  95 outcomes

16. The evaluator facilitated the harvesting of outcomes and followed up through shared google documents, and skype afterwards
Emphasise the time required for ping-ponging

17. Example google docs

18. Example outcome format

19. Principle 7: Rigorously aim for credible-enough outcomes
5. Substantiation

20. No substantiation with third parties, but:
The outcome descriptions concern observable facts and are specific and concrete enough to be verifiable.
They were provided by informants who were knowledgeable about the outcomes.
The informants agreed to go on record with the information they provided about outcomes. 

The relationship between how Family for Every Child contributed and the outcomes was judged by the evaluator to be plausible.

There was cross-validation between secretariat staff and members
After the Outcomes had been gathered, the evaluator together with the users decided that substantiation with third parties was not necessary, since the outcomes were only going to be used internally. Outcomes were the contribution was not clear or the description insufficient were deleted

21. Principle 8: Strive for less because it will be more
Principle 9: Reveal patterns of social change
4. Analyse and interpret

22. 22 outcomes indicating a clear and successful transition of responsibilities from the secretariat and EveryChild to the members of the network represented in the Board

24. NATURE OF THE CHANGE Change in Relationships Change in Activities
Majority of outcomes are about changes activities and capacity (67%)
19% directly linked to changes in relationships
10% between changes in activities and relationships
19%
10%
Change in Activities
Change in Capacity
67%

25. Principle 10: Learn Outcome Harvesting experientially
5. Support use of findings and make strategic decisions
3. Engage network members, board and staff

26. Family members and staff:
Were introduced to the Outcome Harvesting methodology through workshops
Learned to formulate outcomes ‘while doing’ during the Outcome Harvest

27. Family for Every Child members and staff:
Need further guidance in harvesting their outcomes until the network is able to continue on its own

28. Thank you!
Goele Scheers Independent consultant