1. NATIONAL MENTAL HEALTH SERVICES COLLABORATIVE Report of Independent Evaluation Presentation – 7 th February 2012 NATIONAL MENTAL HEALTH SERVICES COLLABORATIVE Report of Independent Evaluation Presentation – 7 th February 2012

2. REPORT OF INDEPENDENT EVALUATION 1.Aims 2.Methods 3.Findings 4.Assessment against objectives 5.Assessment of collaborative processes 6.Recommendations

3. 1. Aims of Evaluation Have objectives been met? How did collaborative elements and processes contribute? Why did these effects happen? What should be done next?

4. 2. Methods a)Site visits, including semi-structured interviews with team members (n=206) b)E-survey (n=117) c)Service user interviews (n=36) d)National steering group/evaluation project group interviews (n=14) e)Project performance data, process documentation, and learning sets

5. 3 a&b. Almost all team members were positive about individual care planning 3 a&b. Almost all team members were positive about individual care planning “Everyone needs a plan so both the staff and the patient knows what’s happening. I don’t really understand why that’s controversial” “We were reluctant at the start for change but we can see the benefits to the clients. The clients love it. It is good for the service. Since the collaborative, we have put the client at the centre. We now have multi-disciplinary input into the ICP. We also have care co-ordinators”

6. 3 a&b. The collaborative had an impact on work with patients, and with each other 3 a&b. The collaborative had an impact on work with patients, and with each other “I have learnt that the old plans were very much focused on people’s problems not their strengths, that was a useful lesson” “It has improved communication, multi-disciplinary team working and documentation. It has motivated people. They are interested and team relationships have improved. Staff knowledge has increased. They used to work in silos. You couldn’t possibly say it had not improved things”

7. 3 a&b. Service user involvement and response were mixed, carers less involved 3 a&b. Service user involvement and response were mixed, carers less involved “There is a service user on the team and this has been really useful and beneficial. This has helped us to look at things from their perspective” “It will take time, we have to start with the service users first and then the carers. Service users were really well involved in the project especially in the early stages, but not so much for the carers, I don’t know why that was”

8. 3 a&b. Objectives began to be met, and team members valued collaboration 3 a&b. Objectives began to be met, and team members valued collaboration “I am pleased that we have made measurable progress in generating the care plan documents and making them work locally. Care planning is now actually ‘happening’ and this is what we set out to do. We have done what we set out to achieve and we are very happy with this” “It has got people talking and communicating. There is more of a drive to implement things”

9. 3 a&b. Some team members looked for more resources, and a firmer compliance process 3 a&b. Some team members looked for more resources, and a firmer compliance process “... some sort of back filling of posts or a dedicated budget to oil the wheels” “Although care planning is a national standard there is no sanction for not completing it and so there is little incentive amidst all the other tasks we have to do”

10. 3 a&b. Many team members look for common documentation; most think progress will be sustained 3 a&b. Many team members look for common documentation; most think progress will be sustained “They should have prepared a sample template that could be adapted for each locality rather than letting every site develop their own document from scratch. There should be a national form, mandated from the centre and compulsory to use” “It will be sustained. Care planning is now core to our work”

11. 3c. Service users vary in their responses to individual care planning “Helps you cope with your illness and your recovery” “Helps with making progress to get out of hospital” “I am not really interested in my care plan. I trust staff implicitly” “I have input in everything but I don’t take the lead. I prefer to take professional advice”

12. 3c. Service user responses suggest some lessons for practice Process more important than format - who is involved, when and where - frequency of reviews Emphasis needed on explaining the purpose and potential of the care plan Encouragement of ownership and responsibility

13. 3d. National leaders valued the collaborative and its achievements “The collaborative has brought us together” Strong will to succeed, and a great deal resting on it Ambition for cultural change beyond individual care planning Urgent need to build on the collaborative’s progress

14. 3e. There was little variation in teams’ sense of positivity; more in their output Care plans – members, reviews, signatures and shares – much variation Recovery-focus : much variation, but mostly improvement

15. 4. Good progress was made against most objectives 4. Good progress was made against most objectives MET-Knowledge review -Evaluation SUBSTANTIALLY-Involvement of service users MET-Generation of change ideas / improvement measures PARTIALLY MET-Involvement of carers -Data collection tools -Quality framework standards NOT MET-Comparing outcomes with non- collaborative sites

16. We found both promoting and hindering factors in the overall process... We found both promoting and hindering factors in the overall process... FACTORS PROMOTING SUCCESSFACTORS HINDERING SUCCESS A topic widely considered to be important Topic complexity, and uncertain focus within the topic Good local teamworkingDiversity of sites (in the context of topic complexity) Effective local project facilitatorsInsufficient stimulation of collaborative activity Effective national project management and leadership Weak senior management support at a local level Good range of training and information materials Limited national clinical leadership Shortened learning sets Late introduction of the key internal measurement return

17. .... With variation at a local level FACTORS PROMOTING SUCCESSFACTORS HINDERING SUCCESS Strong and constructive service user involvement Lack of protected time for facilitators and team members Strong and constructive carer involvement Insufficient buy-in from key team members Focus on teambuilding and team functioning Limited communications within team as to national and local objectives Local lead with clear delegated authority Lack of administrative support Emphasis on compliance issuesUnclear local project planning

18. RECOMMENDATIONS Press on! Multi-professional practice guidance, and care planning documentation Training Data collection Senior level sponsorship and involvement