Outputs, outcomes and impacts Using Theory of Change to plan, develop and evaluate your initiative

To make explicit the link between vision, planning and evaluation To introduce you to Theory of Change as a tool for understanding, developing and evaluating your initiative To help you prepare for your team planning time this afternoon 2 Aims of the session

Theory of Change is… an outcomes-based, participatory method…for planning, evaluation, and organisational capacity-building; it defines all building blocks required to bring about a given long-term goal( a development of programme theory and logic models, adapted to complex change where both the path and the destination are evolving (Gamble 2008) useful as a bridging tool that creates provisional stability, linking planning and evaluation (Saunders et al 2005) 3 Introduction to Theory of Change

Purpose of Theory of Change 4 Plan Formulate your Theory of Change Evaluate Test your Theory of Change Improve Develop your initiative

1. Identify your long-term goal. 2. Conduct backwards mapping to identify the intermediate outcomes (or preconditions) necessary to achieve that goal. 3. Identify the activities that your initiative will undertake to create these outcomes. At the same time consider contextual factors, assumptions, resources needed etc… 5 Developing your Theory of Change

A good Theory of Change should be: Plausible – the logic of the theory is credible Doable – achievable with the resources (and time) available Meaningful – stakeholders see the goals as important and worthwhile Testable – there are credible ways of discovering whether the predicted results occur (and how/why) Developing your Theory of Change 6

In teams, develop a simple Theory of Change for your initiative using the template provided (40 mins) Consider: Is your goal meaningful? Does the logic of your theory make sense? What assumptions are you making? What resources will you need? How will you test your theory? Activity 7

8 Testing your Theory of Change Impacts: Sphere of interest Outcomes: Sphere of influence Ouputs: Sphere of control 'complex' long-term evaluation quantitative and qualitative data, synthetic methods, multiple perspectives 'complicated' medium-term evaluation mainly qualitative data/methods (how & why) 'simple' short-term evaluation mainly quantitative data/methods

1. Data from evaluation activities surveys, focus groups, interviews, case studies etc time-consuming to collect and limited in scope 2. Data generated by initiative itself planning documents, decision logs, network maps, meeting notes, s etc freely available but of limited worth externally 3. Naturally occurring institutional data NSS, module feedback, retention/achievement data etc valued as impact data but difficult to show causality 9 Sources of data

1. What aspects of your initiative will you evaluate? Not everything can or should be evaluated – what is both important and testable? 2. Why are you evaluating? To meet performance indicators? To show impact? To understand, inform and develop your initiative? 3. Who are you evaluating for? Senior managers? Funders? Staff? Students? The sector? Yourselves? 4. How will you evaluate? What sources of data will you use? What methods will you use to collect and analyse the data? 10 Developing your evaluation strategy

Theory of Change makes explicit what is assumed or tacitly understood about your initiative Theory of Change is a planning tool that helps you evaluate and an evaluation tool that helps you plan Your Theory of Change is just that – a theory: it should be revisited and revised throughout your initiative Use the team planning time this afternoon to develop your ToC further and consider your evaluation strategy (purposes, audiences, data and methods) 11 Summary and next steps

Dozois, E, Langlois, M & Blanchet-Cohen, H (2010) DE201: A practitioners guide to developmental evaluation, J W McConnell Family Foundation.A practitioners guide to developmental evaluation Gamble, J (2008) A developmental evaluation primer, J W McConnell Family Foundation.A developmental evaluation primer Rogers, P J (2008) Using Programme Theory for Complicated and Complex Programmes, Evaluation, 14 (1), Using Programme Theory for Complicated and Complex Programmes Saunders, M, Charlier B, Bonamy, J (2005) Using evaluation to create provisional stabilities: Bridging innovation in higher education change processes, Evaluation, 11 (1), Using evaluation to create provisional stabilities: Bridging innovation in higher education change processes Theory of Change Community, 12 References and further reading