1. Monitoring and Evaluation in Asset Based Approaches

2. Workshop outcomes Explain Theory of Change
Provide examples of methods and tools
Explore ways of providing an evidence base
Practical examples of demonstrating impact

3. Some evaluation terms Outputs Inputs Need Activities Outcomes
In groups – give out 5 post it notes and get them to come up with a definition of each term.
Activities
Outcomes

4. Some evaluation terms
Need – problem you are solving Input – what goes in Activity – what you do Output – immediate results Outcome – the change

5. Theory of Change Process Impact Need Inputs Activities Outputs
Outcomes
Process
Impact

6. “We had 100 young people attend our community arts sessions…”
Outputs v Outcomes
“We had 100 young people attend our community arts sessions…”
“So what?”

7. Need
Output
Inputs
Activity

8. The question……… Has my need been met???

9. Two potential outcomes – that we don’t know until we try the cake
Outcome – delicious cake
Outcome – inedible cake

10. If you want to understand the difference you are making…
If you want to understand the difference you are making…. and if you are meeting needs….. you have to measure the outcomes/changes you are making!

11. But it is a challenge….. In groups Draw a person in the middle
From your experience
Thinks of all of the different things that might influence a persons confidence…

12. Wider society
Community
Family and Friends
Individual

13. Tools Tools can be broadly grouped into two categories:
Quantitative tools are generally ones that collect data you can quantify (count!) – they are really useful for telling you what has happened in a project or activity
Qualitative tools are generally ones that produce narrative data – they are really useful for telling you why this has happened and how

14. Types of Tools The tools are:
Cohort survey – ask the same group of people the same questions at different time points
Individual tracking survey – ask the same person the same questions at different time points
Point in time survey
Observation
Interviews
Stories

15. Cohort Survey About the Tool
You ask the same group of people the same questions at two different points in time
Advantages
It is an easy way to collect information from groups. You can use a variety of tools with the group and it is time efficient to input or analyse. It doesn’t matter if the groups are slightly different at the different time points.
Disadvantages
Surveys only show you what has happened not how or why. With a cohort survey you are only tracking the change in the group not individuals

16. Cohort Survey How to do it Basic
The easiest way to do a cohort survey is to ask the group in one go. This can be done by a paper survey. But it is equally easy to do as a group exercise. For this type of survey you are interested in the progress of the group not individuals. So you can put the questions on flip chart on the wall, use voting buttons or ask people to self select where they stand as a group.
Advanced
If you chose questions used in national surveys you can compare your answers to the national picture, You could also use a control group—a similar group of people not included in your project and compare your survey results to this group.

17. Individual Tracking Survey
About the Tool
You ask the same person the same questions at two different time points
Advantages
It is a robust way of understanding and tracking change. It allows you to understand how each individual is progressing in the group. You can use workbooks or personal development plans as an easy way of capturing start points and end points.
Disadvantages
It can be very time consuming, particularly if working with larger groups. You need a mechanism for tracking people so it means asking people for personal information or a unique identifier.

18. Individual Tracking Survey
How to do it
Basic
You need to get responses from people individually. This can be done face to face or via a survey that is distributed to the group. You are likely to need to use some form of paper (or tablet) survey to allow you to do this. You also need a way of tracking individuals to match the surveys up. You are likely to need some form of software (such as Views) to help with this.
Advanced
If you chose questions used in national surveys you can compare your answers to the national picture, You could also use a control group—a similar group of people not included in your project and compare your survey results to this group.

19. Point in Time Survey About the Tool
You ask people once to reflect on the outcomes they have experienced from your project
Advantages
It is easy to administer as you only do it at one point in your project. It gives you some indication of the outcomes people feel they have achieved.
Disadvantages
People will often over claim the outcomes they have achieved using this method. You will also often miss some of the changes in people depending on when you do the survey. You often need to use a survey to do this or some way of ensuring people can respond privately.

20. Point in Time Survey How to do it Basic
Use a reflective baseline. Ask people to score on a scale where they felt they were at the start of the project and where they feel they are now. When you input the data you can use the scales to help quantify the distance people report they have travelled.
Advanced
If you chose questions used in national surveys you can compare your answers to the national picture, You could also use a control group—a similar group of people not included in your project and compare your survey results to this group.

21. Observation About the Tool
The person leading or observing the sessions tracks progress of individuals or the group
Advantages
It is easy to do and requires little time from participants in the group. It allows the observer to consider changes in individuals and the way the whole group interacts. They work well with groups that struggle with surveys and with participants who may not recognise the changes in themselves
Disadvantages
You are making a judgement on the change that has happened in people and so it is your opinion rather than the participants. The observer needs to be systematic so it takes time from the group leader. It can take time to record and analyse data.

22. Observation How to do it Basic
At the end of each session jot down your observations on how the group or individuals have behaved. Do this at the end of every session against a common set of criteria, for example attitude, skills and behaviour of the group. It needs to be done regularly in order for you to be able to track the change.
Advanced
There are several published observational tools. These encourage you to consider the progress against a structured set of criteria, normally based on academic research.

23. Interviews About the Tool
You ask people a set of questions about involvement in the project.
Advantages
It is easy to do and gives you key insights into why or how your project is working. They provide lots of information. You can use interesting tools like video or pod casts to collect and then share what participants say. They can be a great way of involving volunteers in the project.
Disadvantages
They are generally conducted one to one and so this can be time consuming and take the group leader away from the session. They generate a lot of information that needs to be analysed.

24. Interviews How to do it Basic
Identify 3 to 5 questions about your project. Use video of audio recordings to speak to a random selection of people attending your session. Once away from the session listen/watch the recordings and identify any common themes from what people are saying.
Advanced
Focus groups allow you to collect information from many participants at once. They produce a different set of answers because of the group dynamic but require more skilled facilitation. As with interviews you can use a core set of key questions to prompt the discussion.

25. Stories
About the Tool
People tell you about their experience of being involved in your project or activity
Advantages
Stories are brilliant. They allow participants to tell you in their own way what they have got out of a project. You can use writing, drawing, film or audio to collect peoples feedback. You can do it formally or informally at the end of sessions. They can generate information you can use in case studies and to showcase with funders as well as giving you insight into your project.
Disadvantages
If you are collecting lots of stories it can generate lots of information for you to analyse.

26. Stories How to do it Basic
Every couple of months get participants to tell you about the Most Significant Change that has happened to them since being involved in the project. Get to know your group and find the technique that works best for them. Review the stories to identify common themes
Advanced
Once you have a selection of stories as a project team discuss which one you think is the best and why. Although this is a purely subjective exercise it helps you to reflect on what matters most to the project and allows you to shape your project to achieve more of it.

27. What is evidence?

28. What is evidence?
“the available body of facts or information indicating whether a belief or proposition is true or valid”
“Assessing the impact of a project, programme or strategy and identifying lessons for the future”

29. What is evidence anyway?
Scientific Method
Anecdote/ Insight
Amy to insert

30. Challenges
In groups
What are the challenges you face collecting an evidence base – particularly but not exclusively around asset based approaches

31. Some examples Place Based Funding – Sport England and Comic Relief
Come Outside! – Natural Resources Wales
Renew Wales – DTA Wales

32. Common Features
All have subsets of projects trying to achieve different things
Tools that allow you to track journeys and different starting points
Looking for commonalities and differences
Process is really important – understanding the features that led to it being a success

33. Top Tips Be Honest Use a variety of tools
Recognise the value of insight
Know your audience
Be confident in your findings
Get some help!