1. Applied Methodologies in PHI V: Evaluation Dave Jenner (EMPHO) Adapted from material provided by Jo Cooke and Jane Dyas (Trent RDSU) Dara Coppell (Nottingham City PCT)

2. Learning Objectives 1.To understand what is meant by evaluation 2.To increase awareness, through examples, of the use of evaluation in public health 3.To increase awareness of the process of evaluation

3. Evaluation What it is? Why is it done … and who for? How is it done?

4. What it is “Evaluation is concerned with assessing an activity against values and goals in such a way that results can contribute to future decision making and or policy” Tones & Tilford, 1994

5. What it is “Evaluation is attributing value to an intervention by gathering reliable and valid information about it in a systematic way, and by making comparisons, for the purpose of making more informed decisions or understanding causal mechanisms or general principles.” Ovretveit.J 1998

6. Stakeholders those involved in delivery and managing the service/intervention those served or affected commissioners/funders

7. Evaluation involves comparing e.g....... a group which gets the intervention v. one which doesn’t (controlled trial)... state of people, populations, organisations, services before and after an intervention... the achievements of intervention compared with the objectives at outset... what is actually done v. a set of standards (audit)

8. What can you evaluate in public health? services - new or well-established one-off interventions policy changes screening programmes media campaigns surveillance systems outbreak investigations communication methods – newsletters, websites, etc IT systems and other tools a training course!... pretty much anything

9. Questions that evaluations can answer What impact is the intervention/service having on the community? Have service users/local community benefited? Are we meeting their needs? Are we reaching the right people? Who are we reaching? Who are we not reaching? Why? Are our services of a high standard? Do they provide good value for money? Which services are well regarded/not so well regarded? How well are we working with partners?

10. Why evaluate? If you can demonstrate results, you can win support and funding If you don’t measure results, you can’t tell success from failure If you can’t recognise failure, you can’t correct it If you can’t see success, you can’t learn from it

11. Case Study: Health Trainers in the East Midlands

12. Some things to think about (1) Every intervention/service should be evaluated Try and build evaluation into the intervention/service as early as possible Be clear and transparent about why you are evaluating it Evaluation time and resources should be in proportion to the overall project You can’t evaluate everything Think about what future funders would want to know

13. Some things to think about (2) Involve people from the start – all stakeholders Keep it simple and (if possible) fun Make sure that everyone is using the same terminology Provide plenty of feedback... to stakeholders and participants Think qualitative as well as quantitative Share all outcomes - positive, negative, unexpected Share findings and learning as widely as possible

14. Some things to think about (3) Understanding why a project is going well/not so well.

15. Learning Objectives 1.To understand what is meant by evaluation 2.To increase awareness, through examples, of the use of evaluation in public health 3.To increase awareness of the process of evaluation