1. Impact assessment framework
Natasha Innocent
Senior Policy Adviser Learning and Skills MLA

2. Background How libraries support learning
Strong roots into communities
Expertise in working in partnership with a range of other partners
Critical role with digital participation
Support the wider benefits of learning
Provide choice and flexibility
Strong links to Ageing Society

3. What does the framework do?
Helps libraries describe the impact of their learning activities on individuals and communities
A common language to talk to policy makers and funders about the role of libraries in supporting learning outcomes
A methodology to support libraries to plan, evaluate and develop learning activity

4. Inspiring Learning
This framework is based on Inspiring Learning commissioned by MLA and launched in 2003
Inspiring learning is based on a broad and inclusive definition of learning
Launched 5 generic learning outcomes
3 more recent generic social outcomes

5. Generic learning outcomes
Knowledge and understanding
Increase in skills
Attitudes and values
Enjoyment, inspiration, creativity
Activity, behaviour and progression
Learning activities in libraries include any combination of the above

7. decide on data collection methods present and use your findings
develop your story
select SMART indictors
decide on data collection methods
analyse your results
present and use your findings

8. Develop your story
NEEDS why you are developing this learning activity and who it is for?
INPUTS AND ACTIVITIES what resources and activities do you need to develop to deliver this service
OUTPUTS AND OUTCOMES what you expect you will achieve

9. Selecting SMART indicators
Indicators are used to measure, simplify and articulate your story
Indicators can be descriptive, performance or efficiency based
Any good indicator should be SMART – simple to implement, measurable, action-focused, relevant and time-bound

10. ENTITLE output indicators
Number of people who have participated in library activities designed to improve their ICT skills
Number of people who have participated in library activities designed to increase their interest in reading and/or improve their reading abilities

11. ENTITLE outcome indicators
These are specific changes in the attitude, behaviours, knowledge, skills and enjoyment of users as a result of library services
ENTITLE has agreed that the key generic outcomes relating to library ICT and reading activity are knowledge and understanding, skills and attitudes

12. How to collect your data
Review the data you already collect to see to see if you already collect evidence of generic learning outcome indicators
This will help you to decide on whether to adapt a data collection method you already use or develop a new method of data collection

13. Data you may already collect
Data source
Collected
when
Does it give evidence of learning?
Comments book
Daily
Yes
Annual library survey
Yearly
Maybe
Evaluation forms
Post project
Letters from users

14. Quantitative research
Requires a large number of respondents to ensure any sample you take is representative and averages meaningful.
Usually collected by questionnaires with multiple choice questions or by structured interviews with a series of closed or multiple choice questions

15. Qualitative research Useful for understanding something in depth
Involves talking at greater length to a smaller group of people
Qualitative information is gathered by face
to face or telephone interviews, focus group discussions, comments cards or books, open ended questions on questionnaires

16. Ethics
It is important that library staff consider ethical or legal issues when undertaking research. Laws vary from country to country but there are shared principles on
Consent
Interviewing children
Confidentiality
Data Protection

17. Analyse your results
Analysis of the data you collect will reveal if the ‘story’ you mapped out at the beginning actually happened. That means you can test your hypothesis
The approach to analyse will vary according to the type of tool you used and the type of data you collected – whether it is quantitative or qualitative

18. Things to think about when analysing data
In interpreting results you need to decide if 64% is a high or low result
The meaning and significance of the results – you may need to group the data into categories depending on who the programme is targeted at
Template 2 provides examples of how to link outputs and outcomes to an indicator, data collection methods and results

19. Present and use your findings
Highlight your key messages in an executive summary
Describe the purpose of the learning activity that you have evaluated
Describe your chosen methodology
Think about how you could effectively present your findings
Conclusions – relate to your original story

20. Going further Make your evidence more robust by
Examining the long-term impact of the activity by re-surveying people to check if there has been any further progress
Share your findings with other libraries locally and nationally to establish a benchmark
Compare the results for users and non-users and identify if any significant differences

21. Advocacy Make the most of the evidence you collect
Advocacy is speaking out to win influence, gain support, recognition, partners and funding
Combine your outcomes, research, policy and statistical evidence to create a powerful argument for the role your library plays in supporting lifelong learning.

22. For further information contact
Natasha Innocent
Senior Policy Officer: Learning and
Skills
MLA Council