Download presentation
Presentation is loading. Please wait.
1
INTRODUCTION TO THEORY OF CHANGE TRAINING
Hi, I’m Lynn and I’m one of your trainers for the day. This is Katie who’ll be training you too. I hope you found us OK today. Starting with the practicalities: - We are not expecting a fire drill today, so if the alarm goes off, please follow the signs down the corridor on our right and down the stairs. - The toilets are back out by the entrance and to the right. - Let me know if you’re too hot or too cold, there is a dial that controls it a little - and I stress a little! - There’s some tea and coffee behind us, feel free to help yourself throughout the day. - Unless anyone has any pressing need not to, would you mind switching off your mobile phones, and on silent if you have to have them on. - And we are pressed for time, so please can you be back from all the breaks promptly. - Please feel free to interrupt with questions as I go along. - And is everyone OK with Chatham house rules? Rachel Wharton and Cecilie Hestbaek London, July 2016
2
introduction Name What you do
The greatest achievements of your organisation What you want from the day Lynn 10.05 Icebreaker So, we’re going to start with an quick getting to know you exercise. I’d like everyone to turn to the person next to you, and find out their 1. name and 2. what they do, 3. the greatest achievements of your organisation (can change ice-breaker to suit group) 4. what they want from the day [PEOPLE CHAT FOR 5 MINUTES AND THEN START PRESENTING BACK] [WRITE ON FLIPCHART WHAT EVERYONE WANTS OUT OF THE SESSION] [GO THROUGH FLIPCHART LIST AND TALK ABOUT WHAT WE AREN’T COVERING]
3
NPC: TRANSFORMING THE CHARITY SECTOR
Increasing the impact of charities eg, impact-focused theories of change Charity Strengthening the partnership Eg, collaboration towards shared goals Increasing the impact of funders eg, effective commissioning Funder Sector Lyn Thank you for your introductions – a couple of words on NPC NPC is a charity consultancy and think tank. Our mission is to help the charity sector achieve its potential to change lives and society. We work with charities – to help them increase their impact, from strategy to impact measurement, and from research to advice We work with funders – to help them increase their impact, from reviewing their programmes to analysing needs. We are also a think tank and have projects which are strengthening the sector (Research, collaboration with other organisations working towards the same goals) Working at the junction between charities and funders gives us unique insight into the section. We understand the challenges on both sides, and this means we can bring them together to work towards shared goals. NPC works at the nexus between charities and funders Consultancy Think tank 3
4
Agenda Introductions Theory of change Understand: concept purpose components 10.00 – 11.15 Creating suitable goals Exercise: Force field and thinking, feeling, saying, doing 11.30 – 12.30 Step by step ToC Understand pitfalls Create a theory of change 1.30 – 3.00 Present your theory of change Understand uses (design & evaluation) 3.15 – 4.30 15mins 1hr Lynn Ok, here’s the agenda for today. It looks like most of the things that you want to cover will be covered. The things that won’t be covered will be x, y and z. But perhaps we can talk about it in the break and I can give you some pointers on those. We’re aiming to finish today by Is everyone ok with that or does anyone need to leave a bit earlier?
5
Concept, purpose and components of theory of change
Session 1 Concept, purpose and components of theory of change Katie & Lynn to present….
6
theory of changE activities è intermediate outcomes è final goals
Katie Charities can sometimes struggle to know how to start a strategy process or how to start measuring their impact. And when they come to us for advice, one of our first questions will always be ‘do you have a theory of change’? We think it’s the vital building block for both of those things. In simple terms, and we will go into more detail later about what exactly it is, a theory of change is like a map describing the change you want to make and showing your route to making that change. A conceptual map of how activities lead to outcomes
7
Benefits of theory of change
Clarifies what your activities aim to achieve and how Provides the case for why achieving intermediate outcomes is important Provides a structure for identifying what to measure Helps you identify what is and is not working Provides a simple, clear way to articulate your mission and impact Katie There are many benefits that a theory of change can give to an organisation Clarifies your activities and what they aim to achieve – so you understand which are important Provides the case for why intermediate outcomes are important - as you’ll know in your work, many interventions rely on a series of changes – ie intermediate outcomes – before a goal is reached Provides a structure for identifying what to measure – breaks it down into steps which can assign tools to Helps identify what is and isn't working – you can used a theory of change as a basis of a strategy review to look at how and in what ways your activities are effective. Provides a simple clear way to articulate your impact – this can be helpful for stakeholders to understand what you do. Increasingly its something that funders are looking for. EXERCISE
8
The importance of outcomes
? Katie Sidney Harris cartoon-physicists. Need to document thought process/assumptions of why you think your activities will achieve your mission Theories of change can be presented with varying levels of complexity Crucially, linking activities to outcomes to mission and causal links. Mentoring Reduction in youth unemployment
9
Definitions exercise Work in pairs
Match the terminology to the definition Katie *Definitions exercise* Working in pairs Match the terminology to the definition Get participants to give examples of each from their own charity. [WRITE DOWN ON FLIPCHART] Now we’ve covered the components which make up theory of change I’ll talk a bit about the different approaches that you can take to it. There are three main approaches: Planning triangles Logic models Outcomes chains (which is also commonly known as theory of change and may be the model you are most familiar with).
10
Components of a theory of change
the overall impact the charity is trying to achieve Final goal changes that the charity wants to happen Intermediate outcomes why you think one outcome will lead to another and cases where it will not Evidence / assumptions what the service provides Outputs internal enablers (eg, the quality of services, relationships and the values and attitudes of staff) external enablers (eg, social, cultural, economic and political factors, laws, regulations, working with other organisations). Enablers what the charity’s services are Activities resources that go into delivering activities Inputs Katie We will not focus much on inputs and outputs as charities are generally good at tracking these. More important in a theory of change is: Absolutely central is the final goal – in other words, what your organisation is trying to achieve. Intermediate outcomes and how these are caused. The evidence that already exists for this causation (e.g. academic research, public datasets, RCTs) OR the rationale behind your assumptions of causation. The things that will help enable your activities lead to outcomes (internal and external). [ACTIVITY] [GROUPS DISCUSS ACTIVITIES, ENABLERS, EVIDENCE, FINAL GOAL, INTERMEDIATE OUTCOMES] [DISCUSS AS A GROUP] (handover to Lynn to discuss different ways of presenting ToC)
11
What’s the right Approach to Logic modelling / theory of change?
Outcome chain Hybrid For communications Narrative Lynn No right and wrong approach – need to consider what it is for.
12
How to represent a theory of change
Planning Triangle Logic Model Outcomes Chain Lynn However you represent your theory of change, it should be supported by a written description.
13
Intermediate outcomes
PLANNING TRIANGLE Final goal Intermediate outcomes Activities Lynn A planning triangle is perhaps the simplest way to approach a theory of change. It is a visual tool that will help you identify your aims and objectives, and the outcomes you want to achieve. The central aim of the planning triangle is to delineate between activities, intermediate outcomes and final outcomes. How to produce a planning triangle: First begin by agreeing your final goal and then add this to the top of the triangle (the small amount of space encourages you to keep it short). Then populate the middle section with intermediate outcomes, asking yourself what needs to change in order to achieve your final goal? What differences will your programme help to make for your beneficiaries? These must all happen because of your intervention and occur within its lifetime. It is also good to use ‘words of change’ when describing your outcomes, such as more, better, less, improved (although in some cases your outcomes may involve keeping a situation stable or preventing it from getting worse). Finally, at the bottom of the triangle you list the activities you will carry out to achieve your intermediate outcomes. Each activity should have a direct link to one or more of the intermediate outcomes. If not, ask yourself why is the activity included? Or is it something that you should be doing?
14
PLANNING TRIANGLE FOR PARENTING CHARITY
Improved life chances for children from disadvantaged backgrounds Parenting Classes Practical 1-1 support to parents Final goals Parents better support children’s learning Intermediate outcomes Improved attendance and behaviour in school Improved attainment of children from disadvantaged backgrounds Improved parenting skills Lynn Activities
15
Intermediate Outcomes
LOGIC MODEL Inputs Activities Intermediate Outcomes Final goal Outputs Lynn A logic model is similar to a planning triangle, but it allows more detail to be added, particularly about how a programme is delivered. Like the planning triangle, logic models are comparatively easy to do. They tend to be especially useful for smaller interventions and in guiding evaluation, as they offer more detail about how a programme works, and can therefore inform decisions about which aspects to measure. How to go about producing a logic model: Start with your final goal and work backwards to the intermediate outcomes. As before, you then focus on activities, but in more detail. Inputs: the resources that go into the programme, including the budget it requires, the staff time you need to invest and the relationships you need to have in place with other organisations (for example, those that will refer people to you). Activities: what you actually do. The logic model gives you space to break this down into different components, so you can describe the assessment process, the steps you take to become familiar with people and build trust, and the schedule of the different activities you deliver. You can also address the issue of how you deliver - think about the unique qualities you bring and what makes your particular intervention distinctive. Outputs: This is the quantity of activity you deliver; for example, the number of users, how many sessions they receive and the amount of contact you have with them. A logic model should describe the key aspects that will make the intervention work - from inputs through to the final goal. However, there are limitations to both the triangle and the logic model. The process can sometimes become a pro forma exercise in which people simply catalogue programme components without careful consideration of cause and effect – or else list too many features, not all of which are fundamental to the change process. In addition, they are not good at showing the dynamic interplay between features of your intervention. Both models create the impression that inputs take place first, followed by outputs and then outcomes; whereas in most programmes, different aspects occur at different points in the process Similarly, there is only limited scope to plot sequences of outcomes and the more subtle aspects of causality.
16
A logic model for school breakfast Club
Teaching assistants (TAs) trained to provide 1-1 support Description of resources deployed and targeting process Inputs Process for identifying children & families who will benefit Healthy breakfasts available Focus on activities/games to enthuse pupils Activities Description of activities Log of contact time Example work plans Minimum of one hour of 1-1 tuition with TA per week Workplans better tailored to individual pupils needs Targeted pupils attend and continue to attend Engagement Attendance records Feedback from pupils, teachers & parents Pupils eat the breakfasts Pupils engage & enjoy a) tailored learning; b) group activities - Pupil behaviour measures (hard and soft) - Teachers perceptions - Mock test scores Intermediate outcomes Behaviour improves More confident in Maths Engagement and enjoyment in maths lessons generally Lynn - Firstly, it enables you to make evidence collection more strategic, less opportunistic. With the model all evidence collection should now be orientated to testing the model. It may even help you to reduce the overall level of data/evidence collection – because it will also tell you what data/evidence you need to collect. - Secondly, there are team-building benefits to building a model like this. You should find that the process of creating a model for a service helps you to tease out any differences of opinion and achieve some consensus about the aims of the service and how these are to be delivered. - Thirdly, it will enable you to monitor service outputs and outcomes during the course of delivery – thereby helping you to take remedial measures. - Finally, and most importantly, by collecting evidence against each link in the chain you will be in a position where you can begin to attribute any change that you see at the bottom to the service you have delivered at the top. In doing this the evaluation model helps you create a persuasive case about impact by showing you how to build a narrative or story about the probable impact of the service. This is because by testing each stage of the model and undertaking a robust examination of the data and other evidence that you have available, you will be able to build a persuasive case as to whether or not any changes in outcomes can be attributed to the service in question. It is extremely difficult to find incontrovertible proof of impact, rather it is persuasive narrative set against available evidence which improves our understanding of what happened when the service was delivered and what, if anything, changed as a result. Contributes to Final outcomes Maths results amongst targeted pupils show improvement Improved Key Stage test scores
17
Components of a theory of change
the overall impact the charity is trying to achieve Final goal changes that the charity wants to happen Intermediate outcomes why you think one outcome will lead to another and cases where it will not Evidence / assumptions what the service provides Outputs internal enablers (eg, the quality of services, relationships and the values and attitudes of staff) external enablers (eg, social, cultural, economic and political factors, laws, regulations, working with other organisations). Enablers what the charity’s services are Activities resources that go into delivering activities Inputs Lynn The focus on an outcomes chain is the final goal, intermediate outcomes, and activities… As you discuss your theory of change, you want to add more items to help you describe the change you want to achieve. Think about the outputs, enablers, evidence and assumptions
18
Outcomes chain Activity Output Evidence Intermediate outcome
Internal enabler Intermediate outcome Intermediate outcome Final goal Lynn The focus is more on the sequence of change and how different activities relate to specific outcomes, which encourages more in-depth thinking about precisely how change occurs. So a theory of change will provide more of the evidence base, but will also show more clearly the causal links between all the outcomes, making it easier to see what is going on. They were devised to deal with complicated interventions, those that are too interlinked to be planned in a table or basic diagram. One of the important things about it, is that there are key features to the process that make the end result good. We will go through this in more detail later, but just briefly, its generally done in a workshop, with trustees, staff, maybe some beneficiaries or other stakeholders. These bring the different perspectives and makes the end result more robust. This is one of the reasons why funders like it— Firstly it shows that you’ve done the work and thought through the programme and how you’re going to measure your impact. But also it presents it in an easy way to understand, with all your activities, outcomes, and evidence for why you think this will work. Once you’ve done a theory of change it becomes easy to articulate the story of your charity and how its going to achieve your goal. In the US, its been quite normal practice for a while for trusts and foundations to ask for a grantees theory of change, and that’s becoming more common here in the UK too. At NPC, with our focus on charities being as effective as possible, we’ve seen the benefits of charities investing the time in a theory of change to get their strategy or their evaluation right and therefore be able to allocate their resources more effectively. There are other ways to do this, but the steps below outline an approach we have found works well. Step 1: Agree the project you want to work on and –especially if you are doing this for the first time– ensure it is sufficiently narrow, local, focused etc. Briefly, what is the project? Who are you planning to work with and what are their characteristics / needs? What is the context? It helps to break down context into individual capacities, interpersonal relationships, institutional setting and the wider infrastructural system. Step 2: Agree the final / long-term goal for the project. Ultimately, what does the project aim to achieve? For the Young Foundation Accelerator Fund this should refer explicitly to educational inequality. Step 3: Think about intermediate outcomes. What factors are associated with achieving your final goal? Ideally this will be supported by both external/academic evidence and consultation with colleagues/stakeholders. What changes in service users’: i) knowledge and/or skills; ii) attitudes and/or iii) behaviours might contribute to your impact? Think of these as the assets you want your beneficiaries to have or the ways through which you are trying to equip your services users to change. A different way to approach it is to think about what your users will be; i) thinking; ii) feeling; iii) saying; iv) doing as a result of your project. Describe things in terms of words of change; to improve, to enable, to maintain, to reduce, to increase Step 4: You can now move on to thinking about inputs and activities section. What are the basic elements of what you are planning to deliver? Describe things in terms of words of activity; to provide, to support, to run, to offer What will it look like? How will you target it? What resources will you need? How much will it cost? (inputs) How will we deliver? What are the key features? What will quality look like? Who/which organisations will you need to work with? Step 5: Next is the outputs section, which focuses on the volume of work. What volume will you deliver? How many service users will be involved? How much time will you spend with service users? How frequently do you expect them to be involved? Step 6: The next section of the theory of change focuses on ‘engagement’ or quality. What kind of user engagement do you need for the project to work? What is a good/effective project? What should delivery look or feel like in practice/‘on the day’? What makes it work? What features make the project particularly effective? What ‘good practice’ have you learned and plan to deploy? What is unique, distinctive or special about your project or organisation? Will the engagement you want really deliver the intermediate outcomes you are aiming for? The last question above is particularly crucial. It is worth reflecting on whether you have answered this fully. Step 7: Are there any ‘enabling factors’ that will affect the project’s success? These are things outside of your control that could be positive or negative. Step 8: Think about assumptions. A theory of change is only as strong as the weakest link in the chain. Look at your theory of change critically. What are you least confident about? This process is sometimes referred to as identifying your assumptions. Your weakest assumptions are your evidence priorities. You may identify crucial operational assumptions like sufficient referrals or the need for capacity in partner organisations. Alternatively, assumptions may relate to more fundamental issues such as better quality teaching leading to better improved outcomes for disadvantaged young people. Activity Intermediate outcome Intermediate outcome Intermediate outcome
19
Outcome chain For A parenting Charity
Address specific problems stopping children from attending school Practical 1-1 support to parents Improved attendance and behaviour in school School-based activities to help children from disadvantaged backgrounds Increased parental commitment to school Improved attainment of children from disadvantaged backgrounds Improved life chances for children from disadvantaged backgrounds Parenting classes Improved parenting skills Parents better support children’s learning Improvement in children’s abilities and temperament Lynn Plots the broad trajectory. Especially helpful for the organisation to agree, communicate and measure they key mechanisms it works through. So particularly useful for agreeing strategy and overall measures for an organisations success. Less good for understanding how a particular service works Different outcomes for different stakeholders Activity Enabling factor Intermediate outcome Long-term goal
20
Advantages of theory of change: outcomes chain
While an outcomes chain is more complicated and takes longer to create than logic models, it has a number of advantages: Focus on sequence of change Focus on how different activities relate to specific outcomes Continual focus on cause and effect Encourages more in-depth thinking Lynn It is a good idea - alongside the diagram - to explain the theory of change in written form. This often helps to summarise your thinking and highlight the key causal mechanisms in your theory of change.
21
Exercise Lynn
22
The charity produces guidance for people with the condition
People are empowered to manage their condition Health professionals are better informed about the condition The charity answers 13,000 calls and gives information Health professionals give better advice to people with the condition The charity understands the support and information needs of people with the condition Feedback suggests attending conferences improves professional practice The charity promotes guidance on their website, which receives 10,000 hits The charity runs training and conferences for health professionals The charity understands the context and needs of health professionals People are better informed about their condition 646 health professionals attend a conference. Lynn This page goes up on screen at end – DON’T GIVE OUT!
23
What is your theory of change about?
Individuals Projects Campaigns Programmes Lynn Organisations Sectors
24
Creating suitable goals Force field exercise
Session 2 Creating suitable goals Force field exercise START AT OR EARLIER Katie So now we’re going to move onto the more practical things about how a theory of change is developed. A lot of the reasons why theories of change are useful are linked to the process, which is almost as important as the end result.
25
Tracker 1 Tracker 2 Tracker 3 Tracker 4 Where to begin? Alice: "Would you tell me, please, which way I ought to go from here?" Cheshire Cat: "That depends a good deal on where you want to get to." Alice: "I don’t much care where." Cheshire Cat: "Then it doesn’t matter which way you go." Katie Backwards mapping An important feature of theory of change is that you don’t start with where you are now, but with where you want to be. The idea behind that is best illustrated with a quote from Alice’s Adventures in Wonderland. [READ OUT LAST BIT OF QUOTE] You need to know where you are going before you can even begin to decide how you are going to get there. The point of this is that you start with the goal and then think through everything that is needed to achieve the goal. If you’re doing a theory of change for a charity or a programme that already exists, it is really easy to think only in terms of what you do, rather than what you should do. But doing the backwards mapping may reveal that you need to change your activities or work closely with others to make sure that your goal is achieved. When charities are first set up they are usually very focused on what they want to achieve. But as organisations grow older, people get more and more wedded to what they actually do and lose sight of what they want to achieve. Therefore it is important to go through the process of thinking about the goal and the best way to achieve the goal every now and again; something which is the starting point of the theory of change process. Backwards mapping is one of the things that people have the most difficulty with—it’s very hard, and even though I’ve done loads of theories of change, I find it very difficult sometimes. Why are we measuring? 25
26
? VISION IMPACT GOAL AIMS MISSION TERMINOLOGY Katie
First a word about goals The process of developing a theory of change should begin with identifying the needs of the group you are working with and setting the final or long-term goal that you want to achieve. The final goal should describe in broad terms the change you want to see in the group you are working with, and therefore, in society. Remember this is also the summary explanation of why your programme exists. It should be as realistic as possible and you should be clear about what your programme is accountable for. Lots of people get confused between impact, goal, mission, aim and vision. This is often because they can sometimes be used interchangeably And sometimes can be used differently so people can be talking at cross purposes. If you’re doing a theory of change for yourself, what’s important is to make sure everyone involves knows what you are talking about. IF PEOPLE ASK Vision: A picture of the "preferred future;" a statement that describes how the future will look if the organization achieves its ultimate aims Mission: A statement of the overall purpose of an organisation. It describes what you do, for whom you do it, and why you do it (may also include Values – what you stand for) Goal(s): Broad, long-term aims that define accomplishment of the mission Aims: The overall goal(s) of the organisation, or what you are going to do (Objectives: Identifiable key targets you intend to reach on your way to your ultimate goal, or how you are going to do it and when) Impact: The overall difference an organisation, programme or intervention makes AIMS MISSION
27
creating suitable goals
An organisation’s goal should guide strategy It should be clear and unambiguous There is a need to balance ambition and realism Gang crime in a community is reduced Older people remain independent for longer Katie So what exactly will change? It’s important to create suitable goals, whether you’re doing a TOC for an organisation or programme. A good question to ask is ‘how will we know if our mission has been achieved?’ If your organisation isn’t ready to form goals, then you might want to do some exercises around this—such as ‘when the charity has achieved all it wants to achieve, in 100 years time, what would you like to be on the charity’s blue plaque’. Or ‘in one sentence, how would you like to be remembered.’ [EXERCISE] [ASK GROUP FOR GOALS FROM THEIR ORGANISATION OR PROGRAMME] [HOW HAVE THEY FORMULATED THEM] Here are some examples of goals: (First goal) any guesses as to what this organisation does? This is actually a small theatre company (Chicken Shed). The point is that this goal could imply different ways of solving the issue – that’s what TOC establishes. (Second goal) any guesses as to what this organisation does? This is an organisation that provides animal therapy to people with life-limiting conditions to increase their independence and resilience. Guide strategy – it matters and has implications for action. Clear and unambiguous – not too complicated. Doesn’t mean the method has to be immediately obvious. Balance ambition and realism: Some debate about this – visions are often ‘a world in which’… which is fine but may be more difficult to break down. Appropriate level of ambition may be slightly different for org theory of change and for programme theory of change. For example, a health charity that we recently works with has a hugely ambitious vision of a world free of this disease, but the goal of their Care Directorate is that ‘People with X get what they need when they need it.’ They deliberately avoided using terms like ‘quality of life’ because the disease is inherently debilitating and life-limiting, meaning that quality of life was seen as an inappropriate goal by some people with the condition. The resulting goal balances ambition and realism. It ended up being a little ambiguous (you have to define what people need and when they need it!), but this prompts the charity to continually challenge itself about what the needs are and whether its programmes meet those needs.
28
Parents’ counselling service
Multiple goals: Parents’ counselling service Counselling Clients’ emotional or psychological difficulties decrease Clients’ capacity for self care increases Parent / Child interaction improves Clients’ ability to support their children's healthy development improves Children’s emotional resourcefulness improves Katie Simple theory of change we did for a counselling service DESCRIBE SLIDE The organisation we did this for were keen to recognise that what some might regard as an intermediate outcome – CLIENTS’ CAPACITY FOR SELF CARE – is actually a goal in itself although it links to another goal – PARENT CHILD INTERACTION IMPROVES
29
The desired programme goal
exercise Force field Starting to think about outcomes rather than outputs. Tempting to focus on services rather than change you are trying to create for your beneficiaries The desired programme goal What are the obstacles/stopping improvement? What does it look like when the obstacle has been overcome? 11.45 Force field exercise useful way of thinking through outcomes – Useful to think about WHAT YOU’RE TRYING TO ACHIEVE and CHAN GES THAT NEED TO HAPPEN [EXERCISE] Work in pairs – I want you to come up a few obstacles and a few ways to overcome them. The obstacles are mainly to help you think about it – the ways to overcome is what we’ll focus on. The solutions should be phrased as OUTCOMES – ie the change you see as a result of it being overcome Think of maybe 3-4 of each. We’ll do this for few minutes, and then feed back First we need to decide what our goal is – does anyone have an example of a goal from your organisation you’d like us to think about? [if not, go with chicken shed goal] Great – so in pairs, ON POST ITS some obstacles and ways of overcoming them – phrased as outcomes. A few mins and be ready to feed back. [5 mins] OK so some obstacles? [Write up and group] And some ways to overcome / outcomes [write and group] Starting with a goal, identify obstacles and ways to overcome them
30
Exercise Thinking, feeling, saying, doing
Think about what your beneficiaries/users will be…. …Thinking …Feeling …Saying Katie 11:45am Here is another useful way of thinking through intermediate outcomes – Useful to think about CHAN GES THAT NEED TO HAPPEN in terms of what beneficiaries or users are thinking, feeling, saying, and doing You can also think about what changes in service users’: i) knowledge and/or skills; ii) attitudes; and/or iii) behaviours might contribute to your impact? Think of these as the assets you want your beneficiaries to have or the ways through which you are trying to equip your services users to change. …Doing ….as a result of your project.
31
Tips for writing outcomes & activities
Describe outcomes using ‘change’ words—what needs to change for the ultimate goal to be achieved? To improve – To increase To reduce – To maintain To enable Important to specify who the change is happening to: Young people improve their social skills Describe activities and interventions using ‘activity’ words—what will you provide which you believe will contribute to the changes required? To provide – To support To run – To offer It can be helpful to specify who is delivering the activities: Support workers deliver training
32
Session 3 Step by step theory of change Understand pitfalls Create a theory of change outcomes chain
33
HOW TO DO A THEORY OF CHANGE
Get the views of lots of people—do you have the right people in your workshop? Lynn This section is not going to take you through How to do a theory of change in a step by step incremental way. I’m going to give you a handout on this, so don’t worry about writing it all down. BUILD BULLET POINTS Theories of change should be done by getting the views of lots of different people— so normally by getting people together in a workshop. Your theory of change will only be as good as the people in a room, so you want to make sure that they represent a wide range of experiences, that they are interested enough to take part, and that they will be open to questioning. There is no golden rule on numbers - recently did one with 6 highly engaged people which worked well and also did one with about a dozen – less successful – not the numbers so much as we struggled to get understanding on key concepts across the group and was quite dominated by a couple of individuals. Might need break out groups if this happens to ensure that everyone gets heard. If you’re doing something sensitive or something with lots of external stakeholders, you might want to adopt Chatham House rules for the workshop so that people won’t talk afterwards about who said what. NEXT BULLET The next important stage is to make sure the goal is right—it can’t be vague. What exactly will change? A good question to ask is ‘how will we know if this has been achieved?’ If there really isn’t much idea about what the goal should be, then you might want to do some exercises around this— such as ‘what will the world look like when the charity has achieved all it wants to achieve, in 100 years time, what would you like to be on the charity’s blue plaque. In one sentence how would you like to be remembered.’ For example NSPCC – end child cruelty Amnesty has 11 goals - including abolish capital punishment, rapid and fair trials for prisoners of conscience. Send a Cow – physical poverty eliminated with all family members eating appropriately and valued equally. Come back to Send a Cow when talking about outcomes a little bit later. This is perhaps the most important consideration. It is possible to do a theory of change for a whole organisation, but it will take time and the diagrams you create will be complex (see box on page 15). Alternatively you can create a theory of change for simpler programmes - even your work with individuals - which will be easier. Some interventions are well-established and developing a theory of change will be relatively straightforward (there may even be examples you can draw from). Exploratory or innovative programmes will require more thought. The next thing is to remember that you are backwards mapping. The question you need to keep asking is what has to happen for this to happen? Try to think in small steps when you are thinking through all the outcomes. There isn’t really a good trick for this, it just takes a lot of work. The reason you want everyone in the room for this is that you want everyone’s perspective, so I normally ask everyone to go through what they think the outcomes are individually and then bring them all together after that. A tip to save you some time later is to make sure you have defined an outcome, REMBER OUTCOMES ARE CHANGES THAT THE CHARITY WANTS TO SEE HAPPEN as not everyone will be used to thinking in terms of outcomes. Another tip is to ask for them all to be phrased in the same way— that will save you having to interpret them later. So say something like ‘your outcome should be phrased as a short sentence like…children’s literacy improves.’ Solar Aid, another organisation that we have worked with outcomes –– strategic outcome -government supports use of solar power, backwards maps to outcome government understand benefits of solar power. Eventual goal is eliminate the use of the kerosene lamp by the end of the decade Its also important that you think about what HAS to happen for the end goal to be achieved – like the previous example This avoids you doing what you think is a nice idea. Send A Cow –outcomes related to goal of ‘physical poverty eliminated and family members eating appropriately’ – 1. improved nutrition for family and 2.reduction in negative cultural practices. Many programmes are processes or system changes; for example, an organisational restructure, a new IT system or a professional development network. These tend to be more challenging to theorise as the link to end users is indirect, while the intermediate outcomes relate to changes for your colleagues or stakeholders. NEXT SLIDE
34
The process should be as useful as the end result
Theory of change Service users Other stakeholders Customers / funders Service managers & staff Does anybody know why there is a wardrobe in the middle of this slide? Remember the IKEA effect - This is the observation that people feel a greater commitment to and ownership of things they have helped to create. We find this is very much the case with theory of change; the more people you involve at the start the stronger the commitment to the finished article.
35
HOW TO DO A THEORY OF CHANGE
Get the views of lots of people—do you have the right people in your workshop? Consider context—what is the issue you are trying to address (sometimes referred to as a ‘problem’ statement)? End goal—How will you know if it has been achieved? Stage of development—organisation and intervention Backwards mapping—what has to happen for this to happen? Outcomes—are they actually outcomes? Are they all on there? Direct and indirect impact on service users Getting it perfect —80:20 rule Lynn This section is not going to take you through How to do a theory of change in a step by step incremental way. I’m going to give you a handout on this, so don’t worry about writing it all down. BUILD BULLET POINTS Theories of change should be done by getting the views of lots of different people— so normally by getting people together in a workshop. Your theory of change will only be as good as the people in a room, so you want to make sure that they represent a wide range of experiences, that they are interested enough to take part, and that they will be open to questioning. There is no golden rule on numbers - recently did one with 6 highly engaged people which worked well and also did one with about a dozen – less succesfull – not the numbers so much as we struggled to get understanding on key concepts across the group and was quite dominated by a couple of individuals. Might need break out groups if this happens to ensure that everyone gets heard. If you’re doing something sensitive or something with lots of external stakeholders, you might want to adopt Chatham House rules for the workshop so that people won’t talk afterwards about who said what. Remember the IKEA effect - This is the observation that people feel a greater commitment to and ownership of things they have helped to create. We find this is very much the case with theory of change; the more people you involve at the start the stronger the commitment to the finished article. NEXT BULLET The next important stage is to make sure the goal is right—it can’t be vague. What exactly will change? A good question to ask is ‘how will we know if this has been achieved?’ If there really isn’t much idea about what the goal should be, then you might want to do some exercises around this— such as ‘what will the world look like when the charity has achieved all it wants to achieve, in 100 years time, what would you like to be on the charity’s blue plaque. In one sentence how would you like to be remembered.’ For example NSPCC – end child cruelty Amnesty has 11 goals - including abolish capital punishment, rapid and fair trials for prisoners of conscience. Send a Cow – physical poverty eliminated with all family members eating appropriately and valued equally. Come back to Send a Cow when talking about outcomes a little bit later. This is perhaps the most important consideration. It is possible to do a theory of change for a whole organisation, but it will take time and the diagrams you create will be complex (see box on page 15). Alternatively you can create a theory of change for simpler programmes - even your work with individuals - which will be easier. Some interventions are well-established and developing a theory of change will be relatively straightforward (there may even be examples you can draw from). Exploratory or innovative programmes will require more thought. The next thing is to remember that you are backwards mapping. The question you need to keep asking is what has to happen for this to happen? Try to think in small steps when you are thinking through all the outcomes. There isn’t really a good trick for this, it just takes a lot of work. The reason you want everyone in the room for this is that you want everyone’s perspective, so I normally ask everyone to go through what they think the outcomes are individually and then bring them all together after that. A tip to save you some time later is to make sure you have defined an outcome, REMBER OUTCOMES ARE CHANGES THAT THE CHARITY WANTS TO SEE HAPPEN as not everyone will be used to thinking in terms of outcomes. Another tip is to ask for them all to be phrased in the same way— that will save you having to interpret them later. So say something like ‘your outcome should be phrased as a short sentence like…children’s literacy improves.’ Solar Aid, another organisation that we have worked with outcomes –– strategic outcome -government supports use of solar power, backwards maps to outcome government understand benefits of solar power. Eventual goal is eliminate the use of the kerosene lamp by the end of the decade Its also important that you think about what HAS to happen for the end goal to be achieved – like the previous example This avoids you doing what you think is a nice idea. Send A Cow –outcomes related to goal of ‘physical poverty eliminated and family members eating appropriately’ – 1. improved nutrition for family and 2.reduction in negative cultural practices. Many programmes are processes or system changes; for example, an organisational restructure, a new IT system or a professional development network. These tend to be more challenging to theorise as the link to end users is indirect, while the intermediate outcomes relate to changes for your colleagues or stakeholders. NEXT SLIDE
36
CHILDREN ARE FALLING BEHIND AT SCHOOL BECAUSE OF PROBLEMS AT HOME
1. First identify your goal and work back to your outcomes Tip: Discussing the CONTEXT or ‘problem statement’ can help you identify your goal Children’s literacy and numeracy improves Children’s overall grades and school work improve Parents are able to support children to learn Children are able to devote more attention to education Lynn EXPLAIN SLIDE So now have identified goal and have worked back to the outcomes. Understand that there will be a variety of outcomes but there will be a logical order in which they should occur. Issues preventing children from learning are rectified Children’s behaviour improves NB Are they actually outcomes? Are they arranged in an order that makes sense? 36
37
CHILDREN ARE FALLING BEHIND AT SCHOOL BECAUSE OF PROBLEMS AT HOME
2. Then add the links between the outcomes Children’s literacy and numeracy improves Children’s overall grades and school work improve Parents are able to support children to learn Children are able to devote more attention to education Lynn Once you’ve got the outcomes up and arranged in a vague sense, that’s when you want to put on the links, and to think through if there is anything that is missing. An important thing to pick up on here is if there are any feedback loops or things that make the change more complicated. Issues preventing children from learning are identified Issues preventing children from learning are rectified Children’s behaviour improves NB Are there any feedback loops? Things that make change more complicated? 37
38
CHILDREN ARE FALLING BEHIND AT SCHOOL BECAUSE OF PROBLEMS AT HOME
3. Now add the activities that lead on from the outcomes Children’s literacy and numeracy improves Charity provides literacy/ numeracy tutoring Children’s overall grades and school work improve Parents are able to support children to learn Children are able to devote more attention to education Lynn At this stage you want to add on the activities that will lead to your outcomes. That normally falls out quite easily if you’ve done your outcomes right. But if you are in an area which you know relatively little about, you might need to do some research first. Issues preventing children from learning are identified Issues preventing children from learning are rectified Children’s behaviour improves Charity provides support for parents 38
39
CHILDREN ARE FALLING BEHIND AT SCHOOL BECAUSE OF PROBLEMS AT HOME
4. Work through any causal links - evidence & assumptions for these? Previous evaluations suggest tutoring alone will improve literacy for c.70% of children Children’s literacy and numeracy improves Charity provides literacy/ numeracy tutoring Children’s overall grades and school work improve Parents are able to support children to learn Children are able to devote more attention to education Lynn Next you want to work through each causal link and ask why you think x will cause y? Is there evidence that this has happened, is it your experience, or are you making an assumption? If you’re using this to present to a funder, or as the basis of an evaluation, quite often theory or changes at NPC are the first step in setting up a measurement and evaluation framework it can be useful to do a literature search and write down all the evidence to back up the case for your intervention. Not necessary to reinvent the wheel – can rely on proven and evidenced interventions that someone else has done. In fact we are big fans of charities sharing their knowledge with each other. Issues preventing children from learning are identified Issues preventing children from learning are rectified Children’s behaviour improves Charity provides support for parents NB What is already out there and proven? Literature review? 39
40
CHILDREN ARE FALLING BEHIND AT SCHOOL BECAUSE OF PROBLEMS AT HOME
5. Finally add any enabling factors Previous evaluations suggest tutoring alone will improve literacy for c.70% of children Children’s literacy and numeracy improves Charity provides literacy/ numeracy tutoring School identifies children who could benefit from intervention Children’s overall grades and school work improve Parents are able to support children to learn Children are able to devote more attention to education Lynn The next stage is to think about what would derail your intervention. This gives you a good basis for thinking about what the enabling factors for your intervention are— what has to happen for the intervention to work. There’s no need to include everything, like the UK has to have a functioning government, or the lights stay on but things that might actually be a risk. – so in this case if the school fails to indentify the correct children the theory of change will not work The point of noting these down is so that you can a) make sure they happen and b) check if they are happening when you are measuring your intervention. As you can probably guess, doing a theory of change can expand to fill the time that you have. But if you have a good facilitator and a decent goal to start with, I’d expect you to be able to get this far in a day’s workshop. It will be quite messy, but fun and it will take some work to refine it afterwards - something I am working on currently following a workshop day with a client Issues preventing children from learning are identified Issues preventing children from learning are rectified Children’s behaviour improves Charity provides support for parents NB What has to happen or what can derail the intervention? 40
41
CHILDREN ARE FALLING BEHIND AT SCHOOL BECAUSE OF PROBLEMS AT HOME
6. To finesse can add on outputs and/or indicators Previous evaluations suggest tutoring alone will improve literacy for c.70% of children Children’s literacy and numeracy improves Charity provides literacy/ numeracy tutoring 150 children are given one term of literacy tutoring School identifies children who could benefit from intervention Children’s overall grades and school work improve Parents are able to support children to learn Children are able to devote more attention to education Lynn You can then go even further and add on outputs, and indicators, but normally only after a bit more work. Facilitation is key: you need someone who is independent (or willing to act independent), logical and not easily phased as they can get messy. It doesn’t necessarily have to be done by someone external, but it can be quite difficult to facilitate a theory of change inside your own organisation— Issues preventing children from learning are identified Issues preventing children from learning are rectified Children’s behaviour improves Charity provides support for parents NB This usually takes more work 41
42
Common pitfalls Overcomplicating models
More than one outcome in the same box Over-claiming outcomes Not following a logical flow Outcomes not specific enough Lynn There are some common pitfalls that I don’t want you to fall into. Go through these quickly
43
PITFALL 1: Overcomplicating
Network of committed volunteers created to help people get and stay online Reduced isolation Volunteers feel good Staff recruit and train volunteers People are encouraged to ask for help and to learn more People are no longer excluded from the benefits of digital participation Volunteers recruit other volunteers in their local area and find people who need help Decrease in digital divide Reduced social isolation Community Cohesion People are matched with someone who can give them the help they need locally Greater uptake of government and council online services Staff work with media to maximise impact of campaign People have access to cheaper goods/ holidays and family through Skype The campaign and the opportunities to get involved are promoted by local media Local voluntary sector training train volunteers and beneficiaries Lynn Do we really need all these outcomes? Really in programmes where it is a simple provision of a skills (Like in this case) then it doesn’t need to be so complicated. It can really be simpler- people who felt isolated because they couldn’t use computers learn new skills. The simple outcome is new skills, and improved confidence, perhaps reduced isolation if we want to go the full length. I want to show you briefly a very simple theory of change for a parent’s counselling service as a contrast– it as 3 final outcomes (might term goals) and 2 intermediate outcomes Ease of use for older people, eg online shopping deliveries More take up of local training services
44
The Dangers of including too Much
Lynn
45
Pitfall 2 : more than one outcome in the same box
Improved physical and mental health and well-being Which one do we want to focus on? Young people develop new skills and improved self confidence These two things are not the same – skills come first. Lynn Having more than one outcome in the same box—that way lies confusion about what is actually happening.
46
Pitfall 3: Over claiming
Peace in Northern Ireland (& the world) Youth club Support to families of prisoners Campaigns Lynn Over claiming outcomes—actually an example from a colleague who worked for a small charity in Ireland – did 3 things And their mission Peace in Northern Ireland – bit of a tall order. So no bringing about world peace please.
47
Pitfall 4: Not following a logical flow
Young person matched to opportunity exercising choice to engage & acquire new skills Young people’s behaviour improves Lynn Not following a logical flow—often the most difficult thing to get right.
48
Pitfall 5: Outcomes not specific enough
Children and young people have better outcomes What kind of outcomes? Learners more likely to make a positive stake in their community and feel part of society What do we mean by this? Child, young people in care are better able to relate to others, deal with challenges & change What kind of challenges and change? And our final pitfall…
49
EXERCISE: Can you identify the pitfalls?
Project Coordinator provides access to a range of volunteering opportunities and supports young people and school staff during the activity Young person matched to opportunity exercising choice to engage & acquire new skills Young people with MLD are better able to relate to others, deal with challenges & change Young people have increased confidence Young person integrated in the wider local community Young person’s self esteem is improved Young person achieves better integration in the community Young person’s behaviour improves Lynn The project matches young people with mild learning disabilities to volunteering opportunities in the local community. By doing this, the project hope to tackle negative stereotypes and discrimination and improve young people’s confidence, well being and independence. Community participation in volunteering opportunities Young person has increased well being, likelihood of leading independent lives and risk of needing more intensive social services in the future reduced Reduced discrimination of young people with MLD Increased awareness of disability and reduced negative stereotypes
50
Theory of change exercise
Working in groups, begin to backwards map a theory of change Tips: Think for a few minutes first One point on each post it Map backwards from the goal Write in sentences Write in lower case Take it step by step Lynn Ok, so for the next hour(?) I want you to get a bit of practice in doing a theory of change. I’m going to break you into small groups and ask you to do your own theory of change. Now this is only for an hour, so you won’t get far, but I want you to think about this, particularly about the backwards mapping and thinking in terms of outcomes. You can either choose a situation that you know, or I have some scenarios here for you to choose from. Then choose a goal—don’t spend too much time on this. Then think through what has to happen for each step to happen. You’re going to be in different rooms, and there are loads of post it notes and pens available for you. So you have until 3.15pm(?) and then if someone from each group can talk us through their theory of change, and what difficulties they experienced in doing it. Don’t forget to refresh your cup of tea first.
51
Review your theory of change
Is it… Meaningful the outcomes are important and understood Plausible the logic of the model is correct Doable resources are sufficient to implement it Testable there are credible ways to discover whether the results are as predicted Lynn Jim Connell and Adema Klem
52
People with cancer have an improved sense of well-being
Pick a goal People with cancer have an improved sense of well-being Disadvantaged children achieve better qualifications in school Disabled people increase their participation in sport
53
Present your theory of change Uses and evidence
Session 4 Present your theory of change Uses and evidence Katie
54
Reviewing your theory of change
Review principles Avoiding pitfalls Meaningful: Are the outcomes important? Will staff and beneficiaries understand it? Doable: Is it realistic? Does it take into account your capacity and resources? Plausible: Are the activities likely to lead to the outcomes? Testable: Can you test it by collecting data to see if the results are as predicted? Is it overcomplicated? Is there more than one outcome in the same box? Are there leaps to over-claiming outcomes? Does it follow a logical flow? Are the outcomes specific enough? Katie 3.15 Great – I hope you enjoyed that – remember we can only expect to get so far in an hour! So we’re going to take some time for each group to report back (10 mins each) What I’m going to ask is that after each group has presented, the other groups be thinking about The principles The pitfalls In a supportive way! Discuss where we think its strong and where it might need a bit more thinking.
55
Using your theory of change
Strategy Reviewing existing strategy Designing a new programme Allocating resources Thinking about your role in the sector Measurement Identifying causal links Identifying what to measure Communicating your impact 3.45 [before change slide] Ok, so now we’ve gone through the theory of a theory of change, and the practicalities of it, I wanted to talk more about the uses of them. What sort of things do people think they would be useful for and why? [WRITE UP ON FLIPCHART] LOOKING FOR: STRATEGY EVALUATION SECTOR BUILDING NARRATIVE BUILDING We think of this under two headings: Strategy – refocus on goal, not activities Designing – getting people with knowledge together to think about how to solve Resources – what’s working and what isn’t Role in sector – help identify your niche / what’s unique Measurement Links – does doing a really lead to b? Is there any evidence of that? What to measure – daunting at first if have lots of outcomes, but helps you think about what you want to prove, and not just what you can count. Communicate – can be quite visual, useful communication tool
56
The cycle of good impact practice
4 Review 1 Plan 2 Do 3 Assess Identify your desired impact and how to deliver it Understand what to measure about your impact, identify the information you need and how to collect it Deliver the work and impact Collect the information you need about impact Make sense of the information you’ve collected about impact Draw conclusions about the impact you’ve had and compare findings against your plan Communicate information about your impact Learn from your findings and use them to improve your work Katie This shows how you can use your theory of change for strategy and planning as part of the cycle of good impact practice
57
Components of NPC’s approach to measuring social impact
Map your theory of change Prioritise what you measure Choose your level of evidence Select your sources and tools Effective measurement framework developed Strategic vision Leadership Case for impact measurement Katie We’ve talked about TOC as a basis for measurement, but what do we actually mean by this? [Explain it’s the first of 4 pillars] Common approach: list desired outcomes and put appropriate measures against them Ignores distinction between a charity’s direct and indirect influence Risks being a wish list that a charity finds too hard to achieve NPC approach: Use theory of change to identify specifically what a charity hopes to achieve through its activities, and come up with appropriate measures Takes more time than listing measures
58
Theory of change is a starting point for measurement
Prioritise outcomes that: you directly influence (rather than indirectly support) are important / material to your mission are not too costly to measure will produce credible data 4.05 We just said that you should only gather data if important, but how do you decide? costly – proportionate Credible – not based on a leading question, based on a good enough sample, able to see diff before and after, and ideally compare and analyse results.
59
CHILDREN ARE FALLING BEHIND AT SCHOOL BECAUSE OF PROBLEMS AT HOME
Prioritise outcomes for measurement Previous evaluations suggest tutoring alone will improve literacy for c.70% of children Children’s literacy and numeracy improves Charity provides literacy/ numeracy tutoring 150 children are given one term of literacy tutoring School identifies children who could benefit from intervention Children’s overall grades and school work improve Parents are able to support children to learn Children are able to devote more attention to education Katie So how might we prioritise outcomes? The ultimate goal of the charity is that children’s grades and school work improve. In order for this to be achieved a number of important outcomes need to occur, including that children’s literacy and numeracy improves, they devote more attention to education, and that their behaviour improves. Issues preventing children from learning are identified Issues preventing children from learning are rectified Children’s behaviour improves Charity provides support for parents NB This usually takes more work 59
60
HOW TO COLLECT YOUR OWN EVIDENCE
Do you already collect this data? Use this data (if it is fit for purpose) Yes No Has anyone already proved the causal link you want to make? Is it really important you need data on this outcome? Yes Yes No Does anyone else collect this data? Can you access this data? Yes Yes No No No Katie 3.55 A brief word on TOC for measurement. We think it’s a great foundational tool for measurement. For evaluation, you need to think about what you want to measure—which this helps with because it shows the important outcomes clearly. You should use the theory of change to judge what evidence you need to gather to build up evidence of impact This is a flow diagram (looks a bit like a TOC again!) for what steps to take? Use research to limit your data collection Don’t collect unless you decide it is really important Develop your own data source
61
Any questions? Rachel Wharton Cecilie Hestbaek Consultant Consultant
Cecilie Hestbaek Consultant Katie 4.15 So, to wrap up, I just wanted everyone to revisit how they think a theory of change could be useful for their organisation. And/or (because I know we have colleagues here) what you think has been most useful about today? [round the table] Great thanks. And before we close, I want to check what we haven’t covered…. And before you go can I ask you all to fill in your evaluation forms. If you’d like some further help from NPC on your theory of change we offer a variety of services from critiquing a theory of change that you’ve done, to holding in-house training, or facilitating a theory of change session, to developing it with you. If you’re thinking about how you can measure your impact, we have a measurement team here at NPC, who can advise you on that. We also have many other events on, and there’s more detail on this in your packs. Thank you very much for your time and have a safe journey home.
62
New Philanthropy Capital – Transforming the charity sector
THANK YOU New Philanthropy Capital – Transforming the charity sector and
63
EXAMPLE PLANNING TRIANGLE
Final outcome/goal Independent lives Positive relationships with families & wider community Intermediate outcomes Higher aspirations Improved self-esteem Better ‘life skills’ 1-1 support by staff members and volunteers Provide offenders with secure housing Run activities which develop close social bonds Coaching & training on specific skills Practice day-to-day living Activities Signpost to other services/support
64
example LOGIC MODEL Inputs Resources/costs to run a pre-school maths breakfast club Description of resources deployed and targeting process Process for identifying children & families who will benefit Healthy breakfast provided Activities Maths-based group games to engage and enthuse pupils Description of activities Log of contact time Example work plans Minimum of one hour of 1-1 tuition with TA per week Workplans better tailored to individual pupils needs Outputs Targeted pupils attend and continue to attend Attendance records Feedback from pupils, teachers & parents Pupils eat the breakfasts Pupils feel engaged in the breakfast club & enjoy the activities Intermediate outcomes Behaviour improves More confident in Maths Engagement and enjoyment in maths lessons generally - Pupil behaviour measures (hard and soft) - Teachers perceptions - Mock test scores This is the evaluation model that I showed you earlier. The way to think about it is as a tool to help you organise an individual project-level evaluation. It has 4 key advantages; - Firstly, it enables you to make evidence collection more strategic, less opportunistic. With the model all evidence collection should now be orientated to testing the model. It may even help you to reduce the overall level of data/evidence collection – because it will also tell you what data/evidence you need to collect. - Secondly, there are team-building benefits to building a model like this. You should find that the process of creating a model for a service helps you to tease out any differences of opinion and achieve some consensus about the aims of the service and how these are to be delivered. - Thirdly, it will enable you to monitor service outputs and outcomes during the course of delivery – thereby helping you to take remedial measures. - Finally, and most importantly, by collecting evidence against each link in the chain you will be in a position where you can begin to attribute any change that you see at the bottom to the service you have delivered at the top. In doing this the evaluation model helps you create a persuasive case about impact by showing you how to build a narrative or story about the probable impact of the service. This is because by testing each stage of the model and undertaking a robust examination of the data and other evidence that you have available, you will be able to build a persuasive case as to whether or not any changes in outcomes can be attributed to the service in question. It is extremely difficult to find incontrovertible proof of impact, rather it is persuasive narrative set against available evidence which improves our understanding of what happened when the service was delivered and what, if anything, changed as a result. Contributes to Final outcomes Maths results amongst targeted pupils show improvement Improved Key Stage test scores
65
EXAMPLE OUTCOMES CHAIN 1
Residents are better able to manage and sustain their wellbeing Development of bespoke peer support training programme Training of peer supporters with experience of mental health problems Coproduced peer support sessions Peer support Participants pursue purposeful activities (e.g. employment, education, volunteering) Peer supporters and participants feel that they are of value Peer supporters and participants recognise that they have agency in their own lives Peer supporters have Increased employability Peer supporters learn new skills (organisation, active listening etc.) Peer supporters are able to reimagine their identity Peer supporters and participants recognise their own skills and abilities Creative courses Health and fitness courses Coping skills courses Short courses Existing symptoms are brought under control Participants change and adapt their lifestyles Participants have improved resilience Participants self-manage Participants can identify the emotions they experience Participants provided with tools and skills to help them manage Participants are provided with tools and skills to improve their self-confidence People are provided with a space to focus on their well-being People have expanded social networks General benefits of group activity Supporting conditions Relationships are equal; responsibility and decisions are shared People feel secure and able to express themselves freely People feel that they are an equal member of a community People are able to learn from the experience of others People feel that their problems are shared and understood Participants feel supported by their peers
66
EXAMPLE OUTCOMES CHAIN 2
67
A theory of change is not just a chart
68
example Logframe Log frames are used in international development
Similar presentations
© 2025 SlidePlayer.com Inc.
All rights reserved.