1. Evaluating human rights-based interventions Prepared by Dr Alice Donald (Middlesex University)

2. Human rights as both end and means Human rights are both: a set of standards and obligations enshrined in domestic and international law, and a source of principles and practical methods which determine how those standards and obligations are achieved

3. Negative and positive obligations Negative obligations – public authorities must refrain from breaching people’s human rights Positive obligations – in some circumstances, public authorities may need to take proactive steps to ensure that people’s human rights are not breached, even if the harm is caused or threatened by a private individual or entity rather than by the authority itself.

4. “Using human rights to achieve human rights” Ensuring that human rights are respected and promoted at each stage of a process or activity Identifying desired outcomes (short-medium term) and impact (longer term) for service users, carers, staff and other stakeholders and relationships between them May involving redefining – from a human rights perspective – imperatives such as ‘value for money’ and ‘efficiency’

5. Human rights-based interventions – common elements Leadership: public commitment to human rights as reinforcing other organisational priorities and values Systematic participation of service users and carers in design both of interventions and evaluations Screening of policies and practices for human rights compliance (‘traffic light’ approach) Integration of human rights standards and principles (FREDA) into decision-making frameworks – balancing rights in an explicit and transparent way Training – tailored to everyday roles and with immediate opportunities to participants to put their learning into practice Informing all stakeholders about their rights and/or duties

6. What do human rights-based interventions seek to change?

7. Outcomes and impact

8. Aims and objectives - competing priorities? Desired outcomes that make sense to the life of a person using a service – e.g. ‘What will change for the better in my life?’ Desired outcomes that make sense to the practitioner – e.g. ‘What are the targets and indicators that I am obliged to report against?’ – ‘What are my priorities as a clinician?’.

9. Makers and beneficiaries of change

10. Stages of evaluation

11. Stages of evaluation – sample questions Baseline – situation before the intervention – available data? Formative – Do target group/s consider intervention relevant, beneficial? Need to adapt or refine activity to meet objectives? Process – Activities carried out as planned? Reached target group/s? Are they satisfied? Summative - outcomes – Have human rights standards and principles become integrated into core areas of policy/practice as intended? Identifiable changes in e.g. knowledge, skills, behaviour, experience? Summative – impact – Changes to organisational culture; relationships between stakeholders; clinical outcomes? In what specific ways are human rights better respected, protected and promoted?

12. Designing an evaluation - checklist Aims – desired longer-term outcome/s of the human rights-based intervention? Objectives – (SMART) activities or outputs by which to achieve aims Baseline – situation before the intervention – available data? Methodologies – qualitative, quantitative, observational? Who will take part? How will service users participate? Need for external expertise? Timescale – formative, process, summative? Ethics – how will you obtain necessary clearance?

13. Possible pitfalls in evaluation? Bias - emotional or professional investment in the success of the human rights-based intervention? Establishing cause and effect – multiple drivers of change? Changes may not be ‘labelled’ as human rights-driven Resource constraints – scale of evaluation should be proportionate to the project itself. Participants’ pre-existing attitudes and feelings about human rights – possibility of both negative and positive responses; need to factor this into design of evaluation tools

14. Example: evaluation of the human rights-based approach at The State Hospital State Hospital in Lanarkshire - high security forensic mental health hospital for Scotland and Northern Ireland. Provides psychiatric care in conditions of high security for people with mental illness who are compulsorily detained under mental health or criminal law. In 2000, decided to use the Human Rights Act as ‘a vehicle for cultural change’ in order to move away from a custodial and punitive regime towards one that put the human rights of everyone – staff, patients, carers and family members – at the heart of the services.

15. Example: evaluation of the human rights-based approach at The State Hospital Human rights training for staff at different levels and in different roles Human Rights Working Group reviewed all policies and procedures using ‘traffic light’ approach to assess risk of non-compliance: – Is the policy/practice legal? – Does it have a legitimate aim? – Is it proportionate? Creation of a forum for staff, patient, and carer participation in decision-making

16. Example: evaluation of the human rights-based approach at The State Hospital Objectives of evaluation included: to understand the process of developing and implementing an effective human rights-based approach and to draw out key learning for replicating this process successfully in other settings to assess the perceived impact of implementing a HRBA, including benefits for patients, staff and carers from the perspectives of all involved. to identify the extent to which human rights outcomes were perceived to have changed as a result of the adoption of a HRBA. to evaluate the extent to which human rights are now respected in practice at The State Hospital. to develop an effective human rights-based approach to evaluation methodology that can be replicated by those introducing a HRBA to their service.

17. Example: evaluation of the human rights-based approach at The State Hospital Evaluation methodology included: Review of documentary evidence and statistical data Review of other legislative obligations with potential impact on development of policy and practice at The State Hospital Survey of existence of key policies and indicators of effective practice, with qualitative and quantitative data supplied by various departments at The State Hospital Semi-structured interviews with internal and external stakeholders Focus groups with staff (clinical and non-clinical) Focus groups with both recently-admitted and longstanding patients and carers

18. Example: evaluation of the human rights-based approach at The State Hospital “The adoption of a human rights-based approach was successful in supporting a cultural change from an institution where rights were largely ‘left at the door’, and with a ‘them and us’ culture, towards an organisation with a more positive and constructive atmosphere …”. Scottish Human Rights Commission (2009) Human Rights in a Health Care Setting: Making it Work - An Evaluation of a human rights-based approach at The State Hospital.

19. Example: evaluation of the human rights-based approach at The State Hospital Outcomes of the HRBA included: increased staff and patient engagement increased work-related satisfaction amongst staff reduced stress and anxiety amongst staff increased satisfaction among patients over their care and treatment a reduction in ‘blanket’ policies and a greater focus on individual patients’ risks an end to the routine use of seclusion as punishment. Scottish Human Rights Commission (2009) Human Rights in a Health Care Setting: Making it Work - An Evaluation of a human rights-based approach at The State Hospital.

20. Example: evaluation of the human rights-based approach at The State Hospital Lessons learned in relation to evaluation Need for baseline study Need to set clear and measurable objectives at the start of the intervention PANEL - participation, accountability, non-discrimination, empowerment and legality. PANEL principles used to: – shape the initial objectives of the evaluation – develop indicators of effective practice – devise questions for semi-structured interviews and focus groups – identify the data required and mechanisms for collection Scottish Human Rights Commission (2009) Human Rights in a Health Care Setting: Making it Work - An Evaluation of a human rights-based approach at The State Hospital.