1. Showing its worth it: Measuring the impact of volunteering @IVRtweets @VolunteeringEng Nissa Ramsay 11 th February 2012

2. Introduction How important is impact measurement to your volunteering? Very Important Not important Im going to sit on the fence

3. Introduction Who here thinks they could measure the impact of their volunteering project? Id be ace at this I could blag it I dont know what youre talking about

4. Today… My vision: To enhance the capacity of the student volunteering sector Specifically, To increase how you value impact measurement To improve your understanding of impact measurement

5. Lesson 1… Always consider who is asking the question, to who, what their relationship is and the context Lesson 2… The best evaluation questions are part of the delivery process: It should not be hard work Lesson 3… Measuring impact is not an exact science but a balancing act between the ideal world information you would want and the reality of working in your role

6. Why bother? You should do this if: You have a genuine desire to improve the volunteer experience You have to as a requirement of your funding You want to apply for funding You want to shout about how amazing your work is Your volunteers are always deserting you You cant convince any one to volunteer for you You should think twice about doing this if: You have no time/money/resources youre willing to devote You are only doing this as a requirement of your funding You have no use/goal for the results

7. Lesson 4… Measuring impact is also about improving the quality of your work and delivery (not just the end statistics) Lesson 5… Measuring impact can help focus your project/ organisation on its core purpose (BUT can also take you off course) Lesson 6… Measuring impact can work on a range of levels: Project, Thematic area, organisation

8. Where to start 1. Theory of change Articulates how, when and why a change happens as a result of the project/intervention = measure your impact by testing the assumptions behind this = show if you create positive outcomes and why YOU have = Can be on a project/programme or organisation level

9. Case Study: Student Hubs… www.studenthubs.org/xwiki/bin/View/main http://www.youtube.com/watch?v=TLXTnc2YNuk

11. A mission statement/overall aim What? A mission statement Describes the overall impact of the programme The bigger picture – you arent measuring this directly Why? This is your impact: it sets the context as to why you are doing this Student Hubs empowers students to create positive social change, now and in the future

12. Specific aims What? Precise statements about change Break down mission in to components and target groups And/or consider your activities – why are you doing all of them and what do they amount to Why? This will help identify target groups and the core components of your work/ impact/beneficiaries Student Hubs will inspire students to take social action

13. Outcome indicators What? Statements of change that we can measure, link to specific aim Achievable and a change measure (perceptions/feelings) Why? This will help decide who to involve in the evaluation, why and what questions you will ask them Output indicators What? Activities and numbers/targets involved Students feel more motivated to be socially active 500 students involved have taken on new volunteer roles

14. The connections Vision Aim 1 Aim 2 Aim 3 Change = Outcomes Activities = Outputs

15. If your aim is: To provide high quality volunteering opportunities Volunteers views on support, management and training Their volunteer manager (approach / availability etc) Perceptions of bureaucracy (e.g. CRB, interviews) Importance of payment of out-of-pocket expenses Recognition received Workload and level of responsibility Whether they see their involvement as meaningful What they see as the impact of their volunteering Whether they perceive their help as being needed Whether they perceive any barriers limiting this

16. Your turn… Group discussion: Choose either -A project one of you are running -Student Hubs Identify -Specific aims: To increase awareness/perception… -Outcome indicators: Volunteers have increased skills -Output indicators: 25 volunteers have been trained in…

17. Lesson 7… Measuring impact has to start with understanding what you are trying to achieve and how

18. Methodologies and tools Surveys Online (e.g. Surveymonkey) Access to a sample group (e.g. email addresses for online) Response rate and representativeness (using incentives) Focus groups Do not have managers in the same room! In-depth interviews Informal approaches (e.g. walking interviews) Observation Volunteer testimonials and case studies Life history interviewing Mapping exercises

19. Involving volunteers in the process Peer research and volunteer involvement in the process Accessing new skills and experiences Ability of volunteers to form different relationships with respondents, as peers Quality of data vs. professional development of volunteers Project guidance and steering groups Volunteer / user perspective (e.g. the development of research tools)

20. Improving the quality of research Gain organisational buy-in and support Ask only what you need to know Avoid leading questions Always pilot surveys Dont cherry-pick respondents but pick randomly Opportunities for tracking / longitudinal research

21. Using toolkits The Volunteer Impact Assessment Toolkit Hard copy and website Assesses impact on volunteers, the organisation, service users and wider community Guidance on tools and methods £29.99 plus postage

22. Limitations, challenges and pitfalls Self-reported nature of satisfaction Unavoidable to a certain degree Can attempt to triangulate the results (e.g. speak to staff, clients) Ensuring a good response rate Can the results speak on behalf of all volunteers if changes are to be made? Working with volunteers with specific resource needs e.g. disability, low confidence, illiteracy, English as a second language Cost and time implications but everyone needs to have the opportunity to participate

23. Taking it forwards: options Undertaking in-houseSub-contracted Greater understanding of specific volunteering context Draws on expertise in volunteering from a wide variety of fields Cheaper option (financially but not in staff time) Requires limited in-house staff time and easy to manage Allows independent verification and increases confidence in results Volunteers may be more honest and at ease when not talking to staff

24. Communication and dissemination Accessibility of findings Full report vs. summary Plain English and awareness of audience Warts and all (honesty and accuracy) Will increase trust in findings Feedback to the respondents and the volunteer body Recommendations / changes need to be communicated as well as reasons for not changing something Review of changes / developments

25. Conclusion How important is impact measurement? Very Important Not important Im going to sit on the fence Who here thinks they could measure the impact of volunteering within a project? Id be ace at this I could blag it Im not really sure

26. The results At the start of my session 92% felt impact measurement was very important 0% felt it was not important 8% felt they were going to sit on the fence At the end of my session 96% felt impact measurement was very important 0% felt it was not important 4% felt they were going to sit on the fence

27. Conclusion At the start of my session: 8% of you thought youd be ace at measuring impact 45% of you thought you could blag measuring impact 28% of you dont know what Im talking about At the end of my session: 10% of you thought youd be ace at measuring impact 73% of you thought you could blag measuring impact 2% of you dont know what Im talking about

28. The results As a result of me running this session today: 28% more people can blag impact measurement

29. Nissa.ramsay@ivr.org.uk 0207 520 8900 www.ivr.org.uk Thank you