1. Enhancing Practice in Work with Offenders: the Role of Evaluation Jean Hine, De Montfort University

2. Workstream 1: Aim To learn: – Better understanding of what evaluation is undertaken, who does it, how, why, with what result – What facilitates/inhibits – What might help To develop – Evaluation guidance/toolkit – Library of examples of evaluation practice

3. Four Workstreams Good and Effective Practice 3. Literature Review 1. Evaluation 4, EPR 2. SEED

4. Interviews across the EU Mapping of evaluation Working with 2 pilot evaluations Co-ordination with workstreams 2, 3 and 4 Development of evaluation guidance Workstream 1: Components

5. Mapping More scoping than mapping. Not comprehensive but does include good range countries, organisations, individuals Complexity – language and meaning – range organisations involved – structuring of work with offenders in different countries

6. Evaluation takes many forms Specific independent research studies MoJ national studies/reviews Monitoring/management information Target assessment Compliance Project outcomes Individual casework Serious case reviews

7. Who evaluates Some countries have strong commitment to evaluation and active programme, some do not see evaluation as important Most countries have some governmental level unit undertaking some data work, some basic statistical, some special centres/units Most countries have some university involvement but approach varies by discipline Some use of independent organisations

8. What is evaluated? Specific programmes and practices – Electronic monitoring, thinking skills, domestic violence, sex offender programmes, etc Much focus on recidivism/outcomes – But different abilities to assess Competition with prison evaluation Generally short term impact

9. What’s not? Limited: Systematic evaluation of general supervision Service user evaluation Detailed analysis of routinely collected data

10. Key learning from Mapping Wide variation Most evaluation is centrally led – Central commitment important to enable – But prioritises central concerns – Tends to be large scale and quantitative There are interesting pieces of work which need to be more widely known General commitment to a desire for more evaluation, sometimes externally driven

11. Facilitators for evaluation Central commitment External pressures – EU – Funders – Government threat Individual enthusiasm

12. Inhibitors for evaluation Cultural/practitioner resistance Limited Resources – Time as well as money Lack of skills/knowledge Lack of interested audience Overestimation of the task Limited awareness of possibilities Lack of clarity about objectives of practice

13. Pilot work Shared process Working together In some ways artificial but beneficial – for pilots – for guidance Useful differences and similarities Two partners

14. Italy NGO, large scale drug rehabilitation community Offending not a direct focus but desistance an anticipated benefit Formative evaluation How to retain more alternative sentencees at end of sentence Mixed approach – Focus groups – Data analysis Highlighted importance of shared language

15. England Social enterprise offering peer support for employability Offending not direct focus but an anticipated benefit Formative evaluation How to improve success of specific project with probationers

16. Common issues Identifying evaluation question Data system limitations Designing manageable project Resources Time frame Insider role tension Support Changing context

17. Pilots’ Focus Group Right time to evaluate Political context Availability of respondents Rigour and confidence Ethical issues Written guidance too ‘academic’ Cop or coach? Critical friend/benevolent challenging Evaluation is time and labour intensive (but worth it!)

18. How does this inform guidance? Clarity Flexibility Range of uses/users Different starting points Making the most of existing data Making evaluation manageable

19. International comparability difficult Differing terms and meanings Differing criminal justice processes Different criteria for sanctions Different legal contexts

20. Example of Recidivism Usually means reconviction Availability of data: – Routine standard collection or not – Data protection laws Definition of conviction – Legal frameworks Time frames – Judicial procedures

21. The Hope Quality evaluation Sound evidence Informed practice Good outcomes