1. Power struggles, politics and relationships: The power of relationships in determining the success or failure of evaluation. AEA Conference, Minneapolis, October 2012 Kate McKegg

2. 2 Who am I? 2

3. When I say evaluation…. What comes to mind? 3 McKegg, K., AEA 2012 Conference prsentation

4. Some common thinking about evaluation from Not for Profit providers It’s time consuming It’s expensive It’s produces long reports that are hard, and often boring, to read! It doesn’t necessarily provide us with answers to the really important questions we have It’s something we feel we have to do – but would rather not! 4 It’s just not that useful! McKegg, K., AEA 2012 Conference prsentation

5. Relationships in programme contexts 5 Managers Programme Staff Administrators Community / Clients Evaluators Funders ECB facilitators Board members Management Staff

6. Power and social relations in evaluation capacity building McKegg, K., AEA 2012 Conference prsentation 6 Community Groups / NFOs Funders Evaluators / ECB facilitators

7. Philanthropy and Evaluation Capacity Building Growing demand from funders for evaluation of investment Variable capacity among Not for Profit organisations to respond Funder investment in Evaluation Capacity Building (ECB) ECB still an emerging practice 7 McKegg, K., AEA 2012 Conference prsentation c

8. ECB MotivationsNeedsContextsResourcesStrategiesReadinessOutcomes 8 McKegg, K., AEA 2012 Conference prsentation c

9. ECB – a social relation? ECB – is a developmental, interactive, participative process Assumes that done well, or effectively, there will be some sort of change in peoples’: – motivation – skills and knowledge – behavior – and ultimately in improved outcomes (social betterment) or performance 9 McKegg, K., AEA 2012 Conference prsentation

10. What constitutes ‘doing well’? Outcomes are important But they are achieved through interaction – in relationship – with others So, ECB success hinges on the quality of the relationship 10 McKegg, K., AEA 2012 Conference prsentation c

11. A quality relationship? Credibility – “I can trust what she says about….” Rational / action orientated and emotional 11 CharacterCompetence Ability to make people feel comfortable to share or discuss issues Honesty Demonstrating interest and care Paying attention Motivated to support (not focused on own needs) Demonstrated skills and knowledge and results A track record Reliability or the repeated experience of links between promises and actions

12. A trusted relationship Develops over time Is a two-way process – it is relational Can be derailed or encouraged It must earned and deserved Built on shared values and principles (Maister et al, 2004) 12 McKegg, K., AEA 2012 Conference prsentation “Relationships are no longer a means to an end. The relationship themselves matter” (Abma, 2008)

13. Evaluative rubrics An evaluative practice that grows relationships and builds shared values Uses group and individual dialogue Requires participative processes and interaction Three key questions: – On what basis will we decide or demonstrate to others that something we are doing is high quality, valuable, worth doing, or important to pursue? – What are the criteria we will use to make the judgments we need to make about quality and effectiveness? – And on what basis will we determine ‘how good is good’? 13 McKegg, K., AEA 2012 Conference prsentation

14. Evaluative rubrics Make the ‘values’ step transparent and explicit Enable genuine evaluative conversations Get to the heart of what quality and value mean 14McKegg, K., AEA 2012 Conference prsentation

15. Acknowledgements 15 Abma, T., & Widdershoven, G. (2008). Evaluation and/as Social Relation. American Journal of Evaluation, 14 (2), 209-225 Chelimsky, E. (1995). The Political Environment of Evaluation and What it Means for the Development of the Field : Evaluation for a New Century: A Global Perspective. American Journal of Evaluation, 16 (3), 215-225. Jane Davidson (2005). Evaluation Methodology Basics, The Nuts and Bolts of Sound Evaluation, Sage. Labin, S., Duffy, J., Meyers, D., Wandersman, A., & Lesesne, C. (2012. A Research Synthesis of the Evaluation Capacity Building Literature. American Journal of Evaluation, 33(3), 307-338. Maister, D., Green, G., & Galford, R. (2001). The Trusted Advisor. Touchstone, New York. Patton, Michael Quinn (2011). Developmental Evaluation, Guildford Press, New York. Preskill, H., & Boyle, S. (2008). A Multidisciplinary Model of Evaluation Capacity Building. American Journal of Evaluation, 29 (4), 443-459. McKegg, K., AEA 2012 Conference prsentation