Showing its worth it: Measuring the impact Nissa Ramsay 11 th February 2012

Introduction How important is impact measurement to your volunteering? Very Important Not important Im going to sit on the fence

Introduction Who here thinks they could measure the impact of their volunteering project? Id be ace at this I could blag it I dont know what youre talking about

Today… My vision: To enhance the capacity of the student volunteering sector Specifically, To increase how you value impact measurement To improve your understanding of impact measurement

Lesson 1… Always consider who is asking the question, to who, what their relationship is and the context Lesson 2… The best evaluation questions are part of the delivery process: It should not be hard work Lesson 3… Measuring impact is not an exact science but a balancing act between the ideal world information you would want and the reality of working in your role

Why bother? You should do this if: You have a genuine desire to improve the volunteer experience You have to as a requirement of your funding You want to apply for funding You want to shout about how amazing your work is Your volunteers are always deserting you You cant convince any one to volunteer for you You should think twice about doing this if: You have no time/money/resources youre willing to devote You are only doing this as a requirement of your funding You have no use/goal for the results

Lesson 4… Measuring impact is also about improving the quality of your work and delivery (not just the end statistics) Lesson 5… Measuring impact can help focus your project/ organisation on its core purpose (BUT can also take you off course) Lesson 6… Measuring impact can work on a range of levels: Project, Thematic area, organisation

Where to start 1. Theory of change Articulates how, when and why a change happens as a result of the project/intervention = measure your impact by testing the assumptions behind this = show if you create positive outcomes and why YOU have = Can be on a project/programme or organisation level

Case Study: Student Hubs…

A mission statement/overall aim What? A mission statement Describes the overall impact of the programme The bigger picture – you arent measuring this directly Why? This is your impact: it sets the context as to why you are doing this Student Hubs empowers students to create positive social change, now and in the future

Specific aims What? Precise statements about change Break down mission in to components and target groups And/or consider your activities – why are you doing all of them and what do they amount to Why? This will help identify target groups and the core components of your work/ impact/beneficiaries Student Hubs will inspire students to take social action

Outcome indicators What? Statements of change that we can measure, link to specific aim Achievable and a change measure (perceptions/feelings) Why? This will help decide who to involve in the evaluation, why and what questions you will ask them Output indicators What? Activities and numbers/targets involved Students feel more motivated to be socially active 500 students involved have taken on new volunteer roles

The connections Vision Aim 1 Aim 2 Aim 3 Change = Outcomes Activities = Outputs

If your aim is: To provide high quality volunteering opportunities Volunteers views on support, management and training Their volunteer manager (approach / availability etc) Perceptions of bureaucracy (e.g. CRB, interviews) Importance of payment of out-of-pocket expenses Recognition received Workload and level of responsibility Whether they see their involvement as meaningful What they see as the impact of their volunteering Whether they perceive their help as being needed Whether they perceive any barriers limiting this

Your turn… Group discussion: Choose either -A project one of you are running -Student Hubs Identify -Specific aims: To increase awareness/perception… -Outcome indicators: Volunteers have increased skills -Output indicators: 25 volunteers have been trained in…

Lesson 7… Measuring impact has to start with understanding what you are trying to achieve and how

Methodologies and tools Surveys Online (e.g. Surveymonkey) Access to a sample group (e.g. addresses for online) Response rate and representativeness (using incentives) Focus groups Do not have managers in the same room! In-depth interviews Informal approaches (e.g. walking interviews) Observation Volunteer testimonials and case studies Life history interviewing Mapping exercises

Involving volunteers in the process Peer research and volunteer involvement in the process Accessing new skills and experiences Ability of volunteers to form different relationships with respondents, as peers Quality of data vs. professional development of volunteers Project guidance and steering groups Volunteer / user perspective (e.g. the development of research tools)

Improving the quality of research Gain organisational buy-in and support Ask only what you need to know Avoid leading questions Always pilot surveys Dont cherry-pick respondents but pick randomly Opportunities for tracking / longitudinal research

Using toolkits The Volunteer Impact Assessment Toolkit Hard copy and website Assesses impact on volunteers, the organisation, service users and wider community Guidance on tools and methods £29.99 plus postage

Limitations, challenges and pitfalls Self-reported nature of satisfaction Unavoidable to a certain degree Can attempt to triangulate the results (e.g. speak to staff, clients) Ensuring a good response rate Can the results speak on behalf of all volunteers if changes are to be made? Working with volunteers with specific resource needs e.g. disability, low confidence, illiteracy, English as a second language Cost and time implications but everyone needs to have the opportunity to participate

Taking it forwards: options Undertaking in-houseSub-contracted Greater understanding of specific volunteering context Draws on expertise in volunteering from a wide variety of fields Cheaper option (financially but not in staff time) Requires limited in-house staff time and easy to manage Allows independent verification and increases confidence in results Volunteers may be more honest and at ease when not talking to staff

Communication and dissemination Accessibility of findings Full report vs. summary Plain English and awareness of audience Warts and all (honesty and accuracy) Will increase trust in findings Feedback to the respondents and the volunteer body Recommendations / changes need to be communicated as well as reasons for not changing something Review of changes / developments

Conclusion How important is impact measurement? Very Important Not important Im going to sit on the fence Who here thinks they could measure the impact of volunteering within a project? Id be ace at this I could blag it Im not really sure

The results At the start of my session 92% felt impact measurement was very important 0% felt it was not important 8% felt they were going to sit on the fence At the end of my session 96% felt impact measurement was very important 0% felt it was not important 4% felt they were going to sit on the fence

Conclusion At the start of my session: 8% of you thought youd be ace at measuring impact 45% of you thought you could blag measuring impact 28% of you dont know what Im talking about At the end of my session: 10% of you thought youd be ace at measuring impact 73% of you thought you could blag measuring impact 2% of you dont know what Im talking about

The results As a result of me running this session today: 28% more people can blag impact measurement

Thank you