1. Creative evaluation tools: Beyond ‘tired and tested’ Karen Gray Willis Newson

2. Agenda Aims for the workshop A very quick introduction to ‘Creative and Credible’ evaluation Practical exercise: Re-imagining ‘tired and tested’ Summary

3. Background: Willis Newson Specialist arts and health consultancy developing and delivering projects in acute, mental health and community settings Long-term collaboration with Norma Daykin, Professor of Arts and Health at University of West of England UWE led research-based evaluations of Willis Newson arts in mental health programmes: Moving On & Arts @Callington Road Collaborative research and evaluation work culminated in 2 year Knowledge Transfer Partnership (2010/12) Decorative glazing by Stuart Low in Older Adult Unit, Callington Road. Commissioned by Willis Newson for Avon & Wiltshire Mental Health NHS Partnership Trust. Photo; Paul Highnam

4. Evaluation: Three key principles Appropriate Credible Ethical All evaluation needs to embody these principles.

5. Appropriate Appropriate for: –Evaluation aims and objectives –Participants or stakeholder group –Project context Some examples: –Focus groups will not give you quantitative health outcomes data –Validated scales may not be useful in projects with few participants –Questionnaires may not be appropriate for people with questionnaire fatigue Creative exercise – useful for aims setting. Image : Willis Newson.

6. Credible Timing Objectivity Robust application and analysable results Some examples: –If you tell focus group participants that the evaluation findings will help secure funding for next year’s project, will they give you the full story? –How can you measure outcomes for participants if you don’t know where they started? Image from a ‘Savouring & Flow’ workshop run by Light Box as part of The Happiness Project. Photo: BS6 Photography.

7. Ethical Respect Confidentiality Professional integrity Some examples: –A participant has given consent for you to use their words, have you told them exactly where those words will appear? –A participant has agreed to be a case study in your evaluation, but then changes his mind. What do you do? Collage created during a focus group exercise designed to explore the impacts of a project on participants.

8. So, why be creative? To think around problems To counter evaluation fatigue To get meaningful responses that you can’t get in other ways To make best use of your skills and those of your participants To reward and engage your respondents To make evaluation more a part of your project “Creativity is not an escape from disciplined thinking, it is an escape with disciplined thinking.” [Jerry Hirshberg] Reflective practice drawings by Elizabeth Jane Grosse for Arts at Callington Road. Commissioned by Willis Newson for Avon & Wiltshire Mental Health Partnership NHS Trust. Image: Willis Newson.

10. Tired and tested : Scenario Jenny is the manager of a pilot arts on referral scheme, currently planning how to evaluate it. The project is council-funded and will operate in three GP surgeries in Plymouth, serving the city’s most deprived communities, including those where English is a second language. Intended as a first step intervention for patients with mild to moderate anxiety and depression, it aims to build self-esteem and confidence, develop social networks within the community and offer a positive distraction. Participants will be referred by their GP for a block of 10 sessions. Two local artists will deliver a weekly art session at each surgery covering a variety of different art forms including painting, creative writing and photography. There will be approximately 10 participants in each group, although this number is likely to vary from week to week. A practice nurse will be on hand to offer advice or support or signpost participants to other resources. Jenny needs to evidence the impact the sessions have on participants and advocate for continued funding. She has been asked to use the Patient Health Questionnaire (PHQ9) to measure levels of depression among participants, but she is also eager to understand the subjective impact that the project has on each of the participants and to explore their experience over the 10 weeks.

11. Tired and Tested: Scenario Think about: Who? –Who will be doing the evaluating? –Who are the participants? What? –What form should it take? What will it look like? When? –When does it need to happen? What are the issues? –Ethics, participant needs, resources, time…

12. In summary… Be appropriate Be credible Be ethical Be creative… Think about this at every stage: –when you are developing tools –when you are collecting data –when you report and share your findings

13. karen@willisnewson.co.uk Professional development opportunity: Evaluating Arts, Health & Wellbeing Initiatives 5 days of training Feb – Apr 2014 Further information and training