1. The ESDinds project Analysing participation in a cooperative research project on values-based indicators Marie Harder, Gemma Burford, et al. Sustainable Development Coordination Unit, University of Brighton

2. Co-inception: the need for ESDinds Initial discussions between academic researchers, an international consultant, and representatives of 10-15 CSOs working in Education for Sustainable Development CSOs identified values-based indicators for project monitoring and evaluation as a priority: making the invisible visible Meet specifications of EU call for proposals while leaving room for interpretation and co-design by participating CSOs

3. ESDinds Project Consortium Four CSOs: Earth Charter Initiative, Alliance of Religions and Conservation European Baháí Business Forum Peoples Theater, Germany Two university-based research groups: Sustainable Development Coordination Unit, University of Brighton, UK Charles University Environment Center, Prague Independent adviser with expertise in the field of SD indicators

4. Co-design: defining goals & activities Face-to-face meetings of whole consortium (every 6 months) Consultative decision-making Independent adviser acts as mediator Reflection on how to improve collaboration PBworks wiki and e-mail list Reporting back to EU

5. Values and indicators First phase: learning about values and outcomes that CSO partners associated with successful projects Then: How would these values be lived? Initial list of indicators suggested by CSOs Iterative development of indicator list Identifying possible measurement methods Planning field visits to test indicators

6. Case study: Sierra Leone YABC project Youth as Agents of Behavioural Change International Federation of Red Cross & Red Crescent Societies (IFRC): Principles & Values Sierra Leone Red Cross Society Promoting a culture of peace and inclusion Agricultural projects and peer education bringing together vulnerable youth from opposite sides ESDinds field visit in February 2010 Consortiums need: Test the indicators! (NOT co-inception with respect to IFRC)

7. Defining research goals and activities IFRC Director seeking effective ways to evaluate impact and measure change in individuals values Youth leaders (members of YABC network) prioritised 10 indicators for testing Indicators incorporated into a workshop to test IFRCs draft toolkit of YABC exercises in Sierra Leone IFRC and SLRCS representatives selected research methods from a menu provided by researchers

8. Methodologies Quantitative: Spatial survey (Never – Sometimes – Always) Secret ballot survey (as above) Structured observation of participation in decision-making and discussion Qualitative: Conventional focus groups Focus groups with theatre performance CSO staff preferred to watch researchers facilitating the exercises

9. Interpretation of data Some data fed back to youth immediately through focus groups (survey findings) Other results discussed in expert group Some results invalid due to: Conformity bias (`following the crowd) Uncomfortable environment (hot sun) Lack of competent female translator BUT Red Cross still learned something important and useful… Less discrimination in teams than villages!

10. Sustainability ESDinds exercises complemented IFRCs draft toolkit very well National Youth Coordinator confident of being able to incorporate them into SLRCS programmes without further input from researchers IFRC has acknowledged values-based indicators as a useful approach for projects elsewhere

11. Assessing participation at each project stage 1. Consultation (Experts present pre- determined issues) 2. Cooperation (Community offers advice, but decision-making rests with experts) 3. Participation (Equal decision-making by experts and community) 4. Full control (Community controls decision-making, experts advise) Ref. Naylor et al. (2002) Soc Sci Med 55: 1173-1187

12. Relationships of participation Participation varied at different stages of the project – this is an overview! Within consortium: generally 3 Equal decision-making (CSOs/research groups) Between researchers and CSO staff: 3-4 Decision-making mainly led by CSO staff Between CSO staff and youth: (generally) 1-2 Very short, centrally organised workshop Large group Language & literacy barriers

13. Lessons learned Participatory process was critical in responding to challenges, e.g. conformity bias in spatial survey Consider all relationships of participation before concluding that research is truly participatory! Youth to become co-evaluators of their own programs: Identifying research needs Defining goals and activities More hands-on involvement in M&E activities

14. Policy implications Who represents the community? Roles of local experts and leaders/elites? Beneficiary populations are diverse Who are the most marginalised people in this project? (age, gender, poverty, etc…) What are the barriers to their participation - and can they be removed? Is co-inception a realistic possibility? How can they be involved in meaningful ways in co-design? More participation = more benefit?

15. Thanks for listening, and please feel free to join our online community! www.wevalue.org www.esdinds.eu Conference 16-18 December 2010 at the University of Brighton Making the Invisible Visible: An Emerging Community of Practice in Indicators, Sustainability and Values