Action for Prisoners’ Families Seminar 25 June 2014 Quality services: ticking all the boxes

Evaluation and family relationship measures Anna Kazimirski

X AXIS LOWER LIMIT UPPER LIMIT CHART TOP Y AXIS LIMIT v EVALUATION AND FAMILY RELATIONSHIP MEASURES ANNE KAZIMIRSKI, NPC NPC - title of the document (can be changed under view/header and footer. Add date here if needed.3 Action for Prisoners’ Families – 25 th June 2014

X AXIS LOWER LIMIT UPPER LIMIT CHART TOP Y AXIS LIMIT TRANSFORMING THE UK CHARITY SECTOR 4 NPC works at the nexus between charities and funders Charity SectorFunder Increasing the impact of charities eg, impact-focused theories of change Strengthening the partnership Eg, collaboration towards shared goals Increasing the impact of funders eg, effective commissioning Consultancy Think tank

X AXIS LOWER LIMIT UPPER LIMIT CHART TOP Y AXIS LIMIT WHY MEASURING IMPACT IS IMPORTANT 5 The Brandon Centre provides counselling and psychotherapy to young people between the ages of 12 and 21 Influences the debate on “what works” Improved services Raises profile Motivates staff Taken from: Rickey, B, Lumley, T and Ni Ogain, E. (2011) A Journey to Greater Impact New Philanthropy Capital. Helps secure funding

X AXIS LOWER LIMIT UPPER LIMIT CHART TOP Y AXIS LIMIT COMPONENTS OF NPC’S APPROACH TO MEASURING SOCIAL IMPACT 6 Map your theory of change Prioritise what you measure Choose your level of evidence Select your sources and tools Effective measurement framework developed Strategic vision Leadership Case for impact measurement

X AXIS LOWER LIMIT UPPER LIMIT CHART TOP Y AXIS LIMIT THEORY OF CHANGE Links activities  intermediate outcomes  final outcomes clarifies what the activities aim to achieve and how provides a structure for identifying what can be measured provides the case for why achieving intermediate outcomes is important 7 A conceptual map of how activities lead to outcomes

X AXIS LOWER LIMIT UPPER LIMIT CHART TOP Y AXIS LIMIT EXAMPLE THEORY OF CHANGE 8 Family therapy Levels of conflict within families reduced Observable reduction in problem behaviours during process Offenders (and families) feel empowered to take responsibility and maintain gains Strengthened family ties & relationships Contribution to reduced reoffending Reintegration into (non criminal) social & family groups

X AXIS LOWER LIMIT UPPER LIMIT CHART TOP Y AXIS LIMIT OFFENDERS’ FAMILY RELATIONSHIPS 9 Commissioned by NOMS to develop toolkit to measure intermediate outcomes of work to improve family & peer relationships of offenders Project going through peer review, to be published later this year Parallel project: RAND Europe, with ARCS (UK) and University of Glamorgan, focusing on arts and mentoring work with offenders

X AXIS LOWER LIMIT UPPER LIMIT CHART TOP Y AXIS LIMIT 10

X AXIS LOWER LIMIT UPPER LIMIT CHART TOP Y AXIS LIMIT PROJECT PROCESS Evidence review – identifying outcomes and toolkits Consultation with providers & commissioners to select outcomes to focus on Development and piloting of toolkit Consultation with providers & commissioners to inform guidance Analysis and reporting

X AXIS LOWER LIMIT UPPER LIMIT CHART TOP Y AXIS LIMIT DEVELOPMENT OF RELATIONSHIPS TOOLKIT Desistance theory: Link between creation/ investment in family relationships / reduction in peer pressure with reduced offending Consultation: Outcomes differed depending on the type of intervention & beneficiary Needed to reach a balance between being prescriptive and allowing for flexibility We researched existing scales and evaluations in each outcome area and prioritised: good validity and reliability; recently developed; brevity. 12 Toolkit: 25 standardised scales, to be used in paper questionnaires for offenders to complete. For each outcome area: short general scales & longer specialised scales.

X AXIS LOWER LIMIT UPPER LIMIT CHART TOP Y AXIS LIMIT TOOLKIT STRUCTURE 13 Child relationships Wider family relationships RAND personal development outcomes Peer relationships Partner relationships Reduced reoffending Parenting

X AXIS LOWER LIMIT UPPER LIMIT CHART TOP Y AXIS LIMIT OUTCOME AREAS Partner relationships Increased satisfaction with partner relationship, reduced conflict and improved commitment Child relationships Quality of relationships with children, children’s well-being and behaviour Wider family relationships Family functioning, resilience, conflict and communication, satisfaction with relationships in the family and quality of relationships Parenting Improved satisfaction with the parent-child relationship and parenting skills Peer relationships Satisfaction with peer relationships and reduction of negative peer relationships 14

X AXIS LOWER LIMIT UPPER LIMIT CHART TOP Y AXIS LIMIT CHARACTERISTICS OF TOOLKIT The questions can be used as monitoring or as part of an evaluation with a ‘before and after’ (pre/post) research design, with a comparison group where possible 15 Before measure After measure Programme / service Before measure After measure “Intervention” group “Comparison” group

X AXIS LOWER LIMIT UPPER LIMIT CHART TOP Y AXIS LIMIT RESOURCES NPC’s Wellbeing measure: NPC’s papers on How to approach evaluation: Theory of Change: How to communicate your results: Profile of six charities who radically improved their approach to impact measurement: Code of Good Impact Practice Online impact marketplace

Action for Prisoners’ Families Seminar 25 June 2014 Quality services: ticking all the boxes