Strategy Evaluation A vital tool for the savvy strategy maker Australasian Evaluation Society Conference September 2011

Outline of the presentation Setting the scene What is strategy and why do we need it? Drawing on the literature and our experiences Reflections and insights 2

Setting the scene Increasing requests to evaluate strategies, from different quarters Understandings of ‘strategy’ and evaluation differs - different expectations Challenges and implications for evaluation practice Growing body of literature + experience to build on. 3

4 Strategy – a working definition “An integrated set of actions, across the organisation, over the long term, in pursuit of a clearly defined goal”

What can a strategy do? l Identify direction of travel l Identify actions and priorities l Guide resources towards priorities l Make sure everyone is going in the same direction l Provide a longer term focus l External and internal communications tool

In reality Defining a ‘strategy’ is difficult  Boundaries are blurred, meanings change -Context dependent -Different functions (vision, co-ordination) -Viewpoints differ -Documentation differs -Activities, initiatives differ. 6

Strategy design + implementation vary  Deliberate, rationale, linear, stable OR emergent, opportunistic, messy, dynamic  Top down and bottom up approaches. Changes occur over time  Change is to be expected, a living strategy should live, breathe and evolve  Political imperatives, competition for resources, leadership and legislative changes. 7

Evaluating strategies – some ideas  Accept strategy as the evaluand (Patton)  Logical incrementalism (Quinn)  strategy is part deliberate, part emergent  comes from all parts of the organisation – cannot plan centrally  emphasis on learning as you go and making adjustments 8

Mintzberg (1996)  Plan: set of objectives and actions (deliberate)  Pattern: patterns emerging from actions (emergent)  Position: identifying market advantage  Perspective: shared worldview, ideology  Ploy: political manoeuvring, ‘outsmarting’. l Key message: focus on what matters most 9

In our experience l Strategy as aspiration – desired state l Strategy as a solution – lever for change l Strategy as a house for existing initiatives – business as usual l Strategy as a system – integration/joined up l Strategy as a means for engaging multiple voices – shared view 10

Evaluating a strategy - example Wellington Regional Settlement Strategy (WRSS) Strategy to coordinate stakeholder efforts to remove barriers and provide opportunities for newcomers Central + local govt, NGOs, business, healthcare, newcomers 33 activities aligned under 7 goals to be delivered within baseline funding Evaluation was internally owned and lead, objectives developed from the bottom-up. 11

Evaluating a strategy (WRSS) cont’d Monitoring mechanisms in place to track implementation (traffic light reports). Existing evaluation focus: -Document strategy development -Capture key achievements and challenges -Identify way forward. Added focus: Governance and decision making Mix of activities Communication and relationships. 12

Evaluation influencing the strategy Lifting the gaze from monitoring Looking beyond the individual activities to the mix, the fit and the complement of the activities Strategy as perspective equally important focus – how did the strategy change the way (settlement) policy and actions were being conceptualised and delivered Changing role of the governance structures – permission and impetus to perform an active role Focus up and focus across. 13

The importance of learning Unrealised Strategy Changes in environment Learning Intended Strategy Realised Strategy

Our experience – why strategy evaluation can be challenging No organisational/collective/shared view of strategic intent of the strategy: -Those involved in developing strategy had moved on - Strategy boils down to series of actions -The inheritors of the strategy want to keep peace – do not want to challenge. 15

Commissioners of evaluation usually want to focus on actions - strategy as a plan -What happened? Were actions implemented? -What are the results? Tracking indicators.  Tension between what was planned and what happened – due to changes in context.  No real ‘owner’ of the strategy – lead agency but co-chairs rotated -Monitoring information was not being used -Unclear as to who will fix roadblocks. 16

Our reflections Where does strategy evaluation offer?  Alignment or redirection of the evaluand  Does the strategy reflect the right things to do?  Were they done right?  How did what was done depart from what was intended and why?  More informed, transparent learning and accountability  Stronger policy influence  Lifting the actors’ ‘gaze’ -i.e. shifting organisational practice and culture. 17