1. A framework for evaluating partnerships
HOW A COMPREHENSIVE APPROACH CAN STRENGTHEN UNDERSTANDING AND APPLICATION

2. Overview Partnerships as delivery of policy programs have long history
Development of effective frameworks and tools been lagging
Evaluators face technical challenges
Within international development also face conceptual challenge
Case example of a Partnership Evaluation Framework
Use of framework can strengthen understanding

3. A brief history and definitional issues
Not a new concept
No single, common definition – confusion (Carnwell and Carson, 2004; Dowling et al, 2004; Caplan et al, 2007)
Different drivers and underlying theories
Relatively recent emergence in international development (Piciotto, 2007)

4. Current evaluation approaches
Few studies to assess performance or effectiveness (Brinkerhoff, 2002; Jorgensen, 2006; Serafin, 2008)
Focus primarily on outputs (Caplan et al, 2007)
No agreement about elements to be assessed – different elements emphasised by different authors

5. Partnership assessment tools
Include some of the elements emphasised in literature
Primarily for developmental purposes
Beneficial (Sunderland et al, 2009)
Do NOT perform well across all elements (Halliday et al, 2004)

6. Partnership evaluation frameworks
Emerged in response to perceived shortfalls
Range of domains + sub dimensions
More comprehensive:
developmental and evaluative
Outcomes
Questions, evidence criteria
Mixed methods

7. Evaluation challenges
Technical - need to adjust methods (Conlin & Stittat, 2006; Jobin, 2008)
Conceptually (international development)
Uptake in short space of time
Broad scale and scope, and more complex
Promotion of sovereignty (Picciotto, 2007)
Still grappling with purpose
Involving partners
Practical participatory evaluation
Transformative participatory evaluation
(Cousins and Whitmore, 1998)

8. Case example
Review - Australia and Indonesia Partnership Maternal and Neonatal Health
New modality
Review – 2010; 2 years into partnership
Effectiveness and clarification of partnership

9. Case example Partners still grappling with concept No shared view
I wanted a tool or framework to further develop the concept and increase understanding
Partnership Assessment Tool (Hardy, 2003) promising BUT…
Frameworks more comprehensive
BUT…

10. Case example Developed a framework based on mix of:
Elements highlighted by the partners
Salient and commonly used features from literature
Drew heavily on Brinkerhoff (2002)
Plus Caplan et al (2007), Jobin (2008),

11. Partnership Evaluation Framework Hind, 2010
PREREQUISITES
STRUCTURE
PROCESS
OUTCOMES
Enabling environment
Political attitudes
Preoccupations and priorities
Commitment
Expectations
Stability of environment
Formal partnership dimensions
Legitimacy
Resources
Responsibilities
Decision-making
Contribution
Compliance
Partnership practice
Nature of Interaction
Capacity development
Mutuality and equality
Effectiveness
Achieving desired results – including Influence on:
Administrative capacity
Service delivery capacity
Community capacity
Responsiveness
Partnership performance
Relationships
Performance of representatives
Conflict resolution
Creating and strengthening success factors
Drivers:
Organisational & individual
Incentives
Obligations
Efficiency
Make good use of partner resources
Benefits
Costs
Added value
Informal partnership dimensions
Social capital
Reputation
Trust
Institutional elements
Common objectives – public policy purpose
Shared governance
Written agreement

12. Areas of assessment Intended to develop assessment areas participatory
Accepted a pragmatic approach
Practical
Contextually responsive
Consequential
(Datta, 1997)

13. Areas for assessment Prerequisites Enabling environment
Extent to which:
partnership is supported politically; has the backing of key stakeholders; there are real opportunities for collaboration; partners tolerate and encourage power-sharing
partnership meets current priorities
partners are able to focus on the partnership given other work/priorities/demands
partners provide leadership; there are champions; partners are willing to adapt; partners are future oriented
there are clear and reasonable expectations; expectations are similar between partners
environment is stable and conducive to collaborative working; key individuals are in place to facilitate partnership; external context is understood
Drivers: organisational and individual
purpose, mandate, vision of partner organisations supports/facilitates partnership
partners have inbuilt organisational culture, processes, systems, demands to facilitate partnership
partners have legal and/or organisational requirements to partner
decision making processes are clear and sufficient to facilitate partnership
partners have processes in place to address lapses in commitment
Institutional elements
there are common, shared goals linked to relevant public policies; partners articulate what they want to achieve from the partnership
formal and informal governance structures are in place and work; processes and structures have been developed collaboratively
the partnership is articulated in writing yet flexible to adapt as required
Areas for assessment

14. Application of framework
Chose all domains and dimensions
Mixed methods
Document review
Semi-structured interviews
Observational visits
Informal interaction with beneficiaries
Workshops
Presentations by partners
Facilitated group discussion
Partner surveys using 5-point ranking scale
Narrative writing

15. Effectiveness of the Framework
Sufficient breadth to:
address issues re: time, resources, data availability
answer all salient evaluation questions
make judgement using most not all dimensions
Could be used flexibly within the local context
Drew on literature and salient features identified by partners

16. Did it help strengthen the understanding of partnership?
Workshops with partners
Opportunity to discuss and reflect on partnership
New insights + promoted broader thinking
Some further development in understanding
Discussions during report drafting
Opportunity to examine the sub dimensions and their relevance advanced thinking of concept
BUT…
More in-depth discussions were limited so potential constrained

17. Conclusion Evaluating partnerships is essential
Still in formative stage
Concept poses a challenge therefore is a challenge for evaluators
The framework was robust
Need to adapt for local situation
The partnership evaluation framework + techniques can help progress concept

18. Partnership assessment tools
The Partnership Analysis Tool: VicHealth:
The Partnership Self-Assessment Tool: Center for the Advancement of Collaborative Strategies in Health;
The Partnership Assessment Tool: Hardy et el, 2003,

19. Partnership evaluation frameworks
Brinkerhoff (2002) – domains linked through flow chart
Caplan et al (2007) – the ‘drivers’: external, organisational + individual contexts
Jobin (2008) –transaction cost theory; allows consideration of counterfactual; social capital