1. Collaboration and Evaluation Morgan Braganza, M.S.W., Ph.D. Student ReThink Research Group

2. Who Am I?

3. Collaboration

4. Enter  Collaboration “Collaboration shows promise for solving organizational and societal problems” (Gray & Wood, 1991, p. 3-4) One “strategy” for building community capacity is “fostering collaborative relations” (Chaskin, 2001, p. 299) Build community capacity = build relationships

5. Enter  Collaboration Social services provide support to those in need  Agencies often work in silos (Braganza, 2009) Individuals fall through the cracks  Too challenging to navigate the system (Leviten-Reid, 2007) Comprehensive community initiatives bring agencies together  Coordinate services  Inter-organizational collaboration  Community member participation (Chaskin, 2001) Collaboration is important in improving community building and improving society “All sectors are responsible for addressing these problems, preferably through a planned and coordinated approach that combines resources and expertise in new and sustainable ways” (Torjman, 1998, p. 25)

6. Why Collaboration? Working collaboratively “just makes sense” for addressing community issues  It raises awareness of other agencies  It brings attention to existing services and gaps in service  Results in finding new and creative ways to develop high quality sustainable services  Better use of scarce resources  Combined efforts to address issues  Combat larger (rather than smaller) issues  Minimize the duplication of services  Shared knowledge and information exchange  The development of best practices

7. But…What IS Collaboration?

8. According to the Literature… a way of bringing together agencies to address social issues to design/offer programs to address these complex issues exchange information which has the potential for action in the interests of a common purpose “…when a group of autonomous stakeholders of a problem domain engage in an interactive process, using shared rules, norms, and structures, to act or decide to act on issues related to that domain'” true collaboration involves being willing to address issues using methods different than would typically be used “more than simply sharing knowledge and information (communication) and more than a relationship that helps each party achieve its own goals (cooperation and coordination). The purpose is to create a shared vision and joint strategies to address concerns that go beyond the purview of any particular party”

9. The “Continuum of Partnerships”

10. But…What IS Collaboration? …In Practice  Collaboration is understood and practiced differently  Commonalities in definition but no two definitions were the same What does this Mean?  If people are working from their own definition they may be disappointed when their experience doesn’t match their expectations  Establishing a common understanding most important

11. Stumbling Blocks to Successful Collaboration

12. Stumbling Blocks Relationships “Missing” people  collaborations often begin through informal conversations may be hard for smaller organizations who are not connected well to others or have few staff resources Staff turnover Not having a “leader” Conflict between members Competing - particularly to acquire funds Members being unable to contribute equally to the collaborative Involvement in collaboratives solely to meet personal goals (i.e. career advancement)  Make others aware of these agendas (Winer & Ray, 1994)

13. Stumbling Blocks Communication and Conflict Not having a plan for how to resolve conflicts Not having rules for communication Not having frequent enough communication

14. Stumbling Blocks Direction Having unclear goals, expectations and limitations Having different values or beliefs Not having timelines established Not having written protocols and documents in place (i.e. logic models, written agreements and contracts outlining rules and plans)

15. Strategies for Success

16. Direction Define “collaboration” and the level of collaboration desired Be clear about the purpose of the collaborative Be clear about the issue(s) being addressed Be clear about goals Be clear about what each member should contribute and roles – define formal “contracts” whenever possible Determine timelines for activities but also for the collaborative (when and why will it be terminated)

17. Strategies for Success Planning for Action Learn the gaps in services Learn what relevant services are already being offered Determine appropriate collaborative members – ensure multisector viewpoints Determine who has the power within the community to influence change

18. Strategies for Success Meeting Management objectives for the meetings should be stated time lines for each objective established rules should be established for how to participate and to make decisions progress of the meeting should be evaluated at the end summaries of each meeting should be developed contributions of members should be given positive reception (Winer & Ray, 1994)

19. Strategies for Success Conflict Management Conflict should be expected Conflict resolution strategies should be designed (Winer & Ray, 1994) Conflicts within collaboratives can be beneficial  engage in “self-reflection”  investigate “basic assumptions” - correct faulty assumptions  glean greater understanding of the perceptions of others (Mai, et al., 2005, p. 108) Determine (with regular check-ins) whether member needs are being met

20. Strategies for Success Freedom  To share ideas  To engage in free thinking and brainstorming  To ask questions  To challenge  To take risks  To clarify the collaborative Have a good balance between planning and action Limit the number of activities

21. SO… Collaboration is risky and can be challenging

22. Collaboration is Risky and can be Challenging To engage in collaboration:  let go of own ideas and values  come to collective agreement (Chaskin, 2001)  let go of power differentials (Chaskin, 2001)  sacrifice time and resources  accept constructive criticism  be honest and transparent Collaborative projects work best when:  project itself is clear  responsibilities of each organization are clear (Chaskin, 2001)

23. So where does evaluation fit in?

24. Plans for projects make sense Figure out  progress made  how well everything is going Evaluation offers evidence of successes Evaluation offers tangible information to inform policy decisions

25. So where does evaluation fit in? How is it related to collaboration? Strengthens collaborative relationships  uncovers strengths, opportunities for growth, best practices and unexpected learnings Greater likelihood of successful projects  stronger programs  stronger community

26. Evaluation Be Critical Ask Questions Challenge Periodically Reflect

27. Sounds easy enough… Social change can get muddy and challenging Challenges to community capacity building:  An unclear understanding of the population(s) of interest  Who will be involved and role (enter  collaboration)  An unclear idea of the outcomes short or long term? individuals, agencies or the community? what are the goals? what are your indicators of success  Outcomes realistic and achievable?  Tracking data? Recommendation = get some help if possible

28. Levels of Evaluation Level 1 Aspects of the Partnership Level 2 Outcomes of Activities Level 3 Impacts on the Community Ideal: do a little of each Reality: Level 3 is very hard to measure – most do levels 1 and 2

29. Evaluation Activities and Updates from Evaluation Sub- Committee

31. Levels of Evaluation Level 1 Aspects of the Partnership Level 2 Outcomes of Activities Level 3 Impacts on the Community Ideal: do a little of each Reality: Level 3 is very hard to measure – most do levels 1 and 2

32. Level 1: Aspects of the Partnership Member Survey Purpose:  Learn what activities, initiatives or efforts are already available in the community and to begin with Alliance members  What has brought members to the Alliance  What may help members feel value in continuing to be part of the Alliance  Feedback from Members about Alliance activities

33. Level 1: Aspects of the Partnership Member Survey  Survey conducted by telephone  6 participants

34. Level 1: Aspects of the Partnership Member Survey Preliminary Findings:  Various reasons for becoming an Alliance Member (personal, community focused, Alliance focused)  Recommendation: Build on these reasons to ensure Members are receiving the benefits they hoped for  Numerous ways time on Alliance would be considered a wise investment (more members, awareness, progress on activities, etc.)  Recommendation: Establish short term and process goals for the Alliance, for activities and the community

35. Level 1: Aspects of the Partnership Member Survey Preliminary Findings:  Members would eager to contribute tangle supports and resources (i.e. financial, data, equipment, etc.)  Recommendation: Assign roles and responsibilities to leverage contributions  Every Alliance member spoke about activities the Alliance could implement beyond current activities  Recommendation: consider ideas for other activities and determine how current or potential activities align with Alliance goals

36. Level 1: Aspects of the Partnership Member Survey Preliminary Findings:  Variable feedback about activities offered by the Alliance but buy in overall  Recommendations: Ensure all Members know about and have buy in for Alliance activities Ensure activities aim to achieve the goals of the Alliance Clearly conceptualize each activity (target audience, logic and theory of change, etc.) Establish evaluation/measurement criteria Assign roles, responsibilities and timelines Have consensus

37. Level 1: Aspects of the Partnership Member Survey Next Steps:  Survey put into an online survey format  Each Alliance member will be emailed the survey for their feedback

38. You remember this? The “Continuum of Partnerships” How to Make progress: 1) Know where you are today 2) Agree about where you want to be 3) Evaluate! 4) Consider, as a group, how to move “up one notch” 5) Measure again at a later point - determine progress 6) Make collaboration an important activity - dedicate time to making it successful

39. Level 1: Aspects of the Partnership Mapping Exercise Purpose:  To capture current partnership information  To strengthen relationships  Consensus on purpose of the Alliance Next Steps:  All Alliance members will be asked to complete this exercise (paper copy or email)

40. Level 1: Aspects of the Partnership Meeting Checkup Purpose:  To capture feedback about Alliance meetings from members  To strengthen meetings  Ultimate goal: members will feel their time has been a wise investment Next Steps:  All Alliance members will be asked to complete this evaluation activity at some or all meetings (to be determined)

41. Levels of Evaluation Level 1 Aspects of the Partnership Level 2 Outcomes of Activities Level 3 Impacts on the Community Ideal: do a little of each Reality: Level 3 is very hard to measure – most do levels 1 and 2

42. Level 2: Outcomes of Activities Lifelong Learning Week Champions Campaign Inventory

43. Collaboration and Evaluation Together…In practice

44. Questions?

45. Works Cited Beckley, T.M., Martz, D., Nadeau, S., Wall, E. & Reimer, B. (2008). Journal of Rural and Community Development, 3 (3), 56-75. Braganza, M. (2009). “Exploring Collaboration between Organizations Assisting Persons Experiencing Poverty”. Opportunities Waterloo Region. Chaskin, R.J. (1999). Defining community capacity: a framework and implications from a comprehensive community initiative. Retrieved January 15 2012 from www.chapinhall.org/sites/default/files/old_reports/41.pdf. Chaskin, R.J. (2001). Building community capacity: a definitional framework and case studies from a comprehensive community initiative. Urban Affairs Review, 36 (3), 291-323. Gray, B. & Wood, D.J. (1991). Collaborative alliances: Moving from practice to theory. Journal of Applied Behavioural Science, 21 (1), 3- 22. Jakes, S. (2003). Community capacity building. Retrieved February 3 2012 from http://www.google.ca/url?sa=t&rct=j&q=collaboration%20and%20community%20capacity%20building&source=web&cd=7&sqi=2&ved=0 CFgQFjAG&url=http%3A%2F%2Fwww.ces.ncsu.edu%2Fdepts%2Ffcs%2Fpdfs%2Fcap.pdf&ei=jo8sT4eWLqLs0gGBnICRCw&usg=AFQj CNG7ESbpY7zKZTR_Y_uGuKBQf7ZWHw&sig2=fuZ2PnkgyojHyqZrww7gkw. Leake, D. & Black, R. (2005). Cultural and linguistic diversity: implications for transition personnel. Retrieved February 4 2012 from http://www.ncset.org/publications/essentialtools/diversity/partIII.asp. http://www.ncset.org/publications/essentialtools/diversity/partIII.asp Leviten-Reid, E. (2007). Reflecting on vibrant communities (2002-2006). Caledon Institute of Social Policy. Retrieved June 6, 2007 from http://www.caledoninst.org/Publications/PDF/612ENG.pdf. No Author. (2012). Dystopia: George Orwell web source. Retrieved February 3 2012 from http://www.netcharles.com/orwell/articles/col- dystopia.htm The Associated Press. (2012). Davos financial group warns of a ‘dystopian future’. Retrieved February 3 2012 from http://www.cbc.ca/news/world/story/2012/01/11/davos-economy-future.html. Torjman, S. (1998). Community-based poverty reduction. Caledon Institute of Social Policy. Retrieved June 6, 2007 from http://www.caledoninst.org/Publications/PDF/260ENG%2Epdf. Westley, F., Zimmerman, B. & Patton, M.Q. (2006). Getting to maybe: How the world is changed. Toronto: Vintage Canada.