1. Learning Link Scotland
Evidencing the Impact a resource to support the voluntary sector to evidence the impact of adult learning
Learning Link Scotland

2. Learning Link Scotland
We are a national membership organisation for voluntary sector adult learning projects in Scotland. We create and deliver services and projects on policy and practice in adult education.

3. Introducing yourself What impact do you want or need to evidence?
How do you do it now?
What do you want to be able to do?

4. Some issues for measuring impact
Knowing where to start
Feeling overwhelmed by options
Time
Architect or archivist?
Linking everything together
One method for multiple funders
Capacity – who is responsible?

5. The Evidencing the Impact resource
The Evidencing the Impact resource builds on the Explaining the Difference pack, published by Learning Link Scotland in 2009 and is a response to calls from the voluntary sector for user-friendly guidance on how to evidence clearly the impact of adult learning on individuals, on communities and on life chances.

6. We aim to provide:
an understanding of the key national policy drivers to provide context for any evaluation
an opportunity to learn about and reflect on the terminology we use to talk about evaluation (such as “output” and “outcome”)
information about the range of evaluation tools that are available, such as the Logic Model or Return on Investment Framework, outlining their structure and where to find them
information about the various quality frameworks that voluntary sector adult learning providers may need to respond to, such as How Good is our Third Sector Organisation?
signposting to further information and guidance about evaluation and evidencing impact.

7. Outcomes Indicator Outputs Impact Activities Measuring Evidence

8. Spotting outcomes Consider the list Identify the outcomes
Can you say why they’re outcomes?
Can you say why others are not?
and, finally …
Share your ideas with others beside you

9. A couple of models we used
Logic Model
Return on Investment Model

10. Logic Model
A logic model tells the story of your project or programme in a diagram and a few simple words. It shows a causal connection between the need you have identified, what you do and how this makes a difference for individuals and communities. You can use logic modelling to plan a project or programme (as an “architectural” approach), or to examine and record ways in which a project or programme has produced outcomes after it has ended (as an “archaeological” approach). The logic model reflects organisations’ particular contexts, purposes, circumstances and focuses of interest. Therefore different organisations’ logic models will vary in focus and might look dissimilar.

11. Logic Model
NEED INPUT ACTIVITY OUTCOME Headache Pills Take pills Get better

14. Online tutorial www.learninglinkscotland.org.uk

15. Logic Model – case study
Emma Whitelock – Lead Scotland (Linking Education and Disability)

16. Planning for Impact
The measurement of “return on investment” usually refers to financial reward that follows financial investment. However, it can be used to measure the success of a project or programme according to any number of indicators that you are seeking. It is particularly useful for planning for impact before or at the start of a project or programme (for example, a pilot learning programme).

17. Planning for Impact How to plan for impact:
Identifying stages of the programme
Identifying outcomes and indicators
Identifying evidence sources

18. Computing for Dads Stage 1: engagement of learners
Outcome:
we want single dads in this area to see the relevance of the computing course to them
Indicators:
60% of the open day attendees sign up for more information
Evidence:
sign-up sheet

20. Planning for Impact– case study
Karen Adams – NHS Education for Scotland

21. What we learned from the Pilots
Feedback from the refresh provided clear learning points that future users of the resource might benefit from reflecting on: the importance of dialogue in the evaluation process – it is not a process to be undertaken by an individual in a room on their own; rather it should actively involve colleagues, service users, partner organisations and others. In the pilot, dialogue appeared to be key to the successful development of some organisations’ logic models.

22. What we learned
the process of evaluation and evidencing impact was often thought by participants to be as, if not more, important than the final documentary evidence – participants said that they enjoyed and found value in the chance to reflect deeply, consider projects from multiple angles, and to engage with colleagues and partners as they did this.

23. What we learned
assumptions you make should be reflected on as a fundamental part of the evaluation process – identifying assumptions you are making can be very challenging; for example, if you are assuming that family learning makes a positive difference to outcomes for children and their parents, then do you have evidence to support this, in case you were asked to provide it? By identifying assumptions you are ensuring you are clear in your justification for a particular project or programme, or for undertaking it in a particular way.

24. Some advice:
Start the process when making funding applications as it will help articulate what you hope to achieve and enable you to get a project off to a flying start with clarity of purpose and how success will be measured.
Share as widely as possible in your organisation both formally and informally. Widespread adoption of the logic modelling will help to maintain clarity of purpose and help to inform stakeholders of the social value of your work.
Evidencing the Impact is a handy gateway to a range of evaluation tools and the legislative framework in which they operate but is not a short cut to demonstrating the impact of your work.
Tim Green, Education Development Manager, South East Scotland

25. Your outcomes
Write down one outcome that you might want to achieve

26. Trying it out
Look at the examples of the Logic model and the Planning for Impact model
Choose one model to follow and use the blank pro-forma and the Outcome you have just identified to start creating your own plan

27. www.learninglinkscotland.org.uk jhowie@learninglinkscotland.org.uk
Find out more