1. The Reading Outcomes Framework Toolkit
Debbie Hicks, Creative Director
ASCEL Conference, November 2017

2. Areas to cover today Overview Pilot phase Feedback and recommendations
Discussion
Next steps

3. What are we trying to achieve?
Understand more about the impact of our reading for pleasure, health and well being programmes.
Use this to drive improvement and make the case for investment.
Develop shared outcomes and measures.

4. Project aims
Collaborate to improve evaluation across the sector through
An outcomes framework that maps the impacts of reading for pleasure and empowerment
An evaluation toolkit that provides advice about how to use the framework and tools to measure the impact of activity that encourages reading

5. Partners

6. Outcomes Framework Handouts: image of Framework
Document with descriptions

7. Evaluation Toolkit

8. Evaluation Toolkit Contents
An online and interactive Evaluation Toolkit that includes:
Guidance on how to use the Framework
Advice on how to conduct an evaluation
Tools (questions) to measure outcomes
Evidence for impact measure

9. Programmes in scope Reading for pleasure and empowerment
Reading for entertainment or self-cultivation, that is done through choice.
All target beneficiaries
All ages (incl. parents/carers answering on behalf of children).
All settings e.g. hospital, prison, school etc.
Emerging and confident readers.
All types of reading material
Fiction, non-fiction, comics, audio books, listening to others read and shared reading.

11. Toolkit

12. Measures – personal outcomes

13. Evidence

14. Evidence

15. In the field
Book Aid International uses the toolkit and thinks it is “amazing”.
Built into SLIC commissioned refresh of the Scottish Reading Strategy which began in Spring 2017.
Used to evaluate the First Minister’s Reading Challenge which engages 74% of primary schools in Scotland.
Literature Works included use of toolkit
in live bids.bi
NPC (leading evaluation consultancy)
recommends it to clients.

16. Pilot Funded by ACE in 2017. Aims
To test how the toolkit works in practice
To evaluate the impact of reading for pleasure and empowerment
To provide case studies
Delivered in 5 different settings
Supported by Renaisi

17. Five Pilot Sites
Accelerated reader programme, King Alfred’s Academy, Oxfordshire
Reading Ahead at Newcastle under Lyme College
Kirklees Home Library Service
Cumbria Libraries author events programme for school children and adults
Castleton Children’s Centre Bedtime reading workshop

18. Case Study: Bedtime Reading Workshop
Selected outcomes for children and parents
Confidence about reading
Sharing enjoyment of reading
Parental engagement in other activities
Agreed tools- post survey for parents
Created survey using a mixture of questions from the tool kit and from previous surveys
Analysed evidence
76% of parents thought children more confident about reading
100% said children enjoyed the reading activity
76% felt more confident about sharing reading with their child
100% said they would attend similar events

19. Other sources of feedback
Consultation with target users at conferences/events.
Online consultation with target users.
Steering group feedback.
Reading Agency staff feedback .
A review of Reading Engagement Measures.

20. General Findings
Valuable to focus on reading for pleasure and empowerment not literacy
Positive response and
understanding of value
and application
Works in a range of settings, with different audiences and different types of activity
Outcomes framework, evidence base and question bank useful

21. The Challenges
Low levels of evaluation confidence across the sector in designing and conducting evaluations, data analysis and reporting.
People need time to understand how to use and navigate it.
Gaps in toolkit relating to outcomes, questions and audiences.
Format and navigation clunky at times.

22. How we want to improve it

23. Discussion and feedback
In groups consider the following questions
What programmes/activities could you use the toolkit to evaluate
Select an activity and think how you would map its aims to the Framework’s Outcomes
What will you need to do/think about when setting up an evaluation
What challenges/difficulties do you anticipate
What tools/resources would you like to see developed as part of any future work

24. Tips

25. Our next steps Publish pilot case studies, including:
Setting and programme
Approach
Findings
Publish new database of
reading engagement measures.
Seek funding for improvements needed, develop further and help target users.

26. Stay in touch