1. Evaluating a whole systems approach to integrated care in North West London Nuffield Trust & London School of Economics 22 June 2015 Holly Holder (holly.holder@nuffieldtrust.org.uk)holly.holder@nuffieldtrust.org.uk Project team: Matthew Gaskins, Holly Holder, Judith Smith and Gerald Wistow

2. Presentation structure 1.Methodological challenges 2.What is the Whole Systems Integrated Care programme? 3.Our methods 4.My reflections

3. Methodological challenges How do you evaluate such a large and complex programme? In such a multi-level programme, which level do you investigate? How do you evaluate something that is constantly changing?

4. What is the Whole Systems Integrated Care (WSIC) programme vision? Vision Care is coordinated around the individual Care is provided in the most appropriate setting Funding flows to where it is needed Aims People will report a better quality of life Quality of care will improve Better care will be delivered at lower cost Providers will operate more effectively Professional experience will improve

5. WSIC process and delivery Develop accountable care partnerships (ACPs) Use a capitation payment model that incentivises providers to work together and focus on population outcomes rather than activity To have GPs at the centre of organising and coordinating care within the ACPs To make patients equal partners in all aspects of design

6. North West London 2 million population 8 local boroughs 8 clinical commissioning groups £4+ billion annual health and social care spend

7. A complex provider landscape Mount Vernon Harefield RNOH Hillingdon Northwick Park Ealing West Middlesex Central Middlesex Hammersmith Charing Cross St Mary’s Chelsea and Westminster Royal Brompton Royal Marsden Imperial London North West Western Eye St Charles 10 acute and specialist hospital trusts 2 mental health trusts 2 community health trusts 400+ GP practices

8. Which organisations are involved in WSIC? Source: WSIC Integrated Care Toolkit

9. How is the WSIC programme funded? Top-slice of 2.5% of pooled budgets of eight NW London CCGs and NHSE commissioning budget for NW London Programme management team also drives other, related initiatives: acute hospital reconfiguration, and programmes to reform primary care and mental health services Additionally, all eight NW London CCGs have joined to work as a collaborative

10. Two levels of operation: pan-NWL level Adapted from WSIC Integrated Care Toolkit Governance structures

11. Activities at a pan-NWL level Tackling the ‘tough nuts’ once at the pan-NW London level... Source: WSIC Integrated Care Toolkit

12. Two levels of operation: local level (Early Adopters) Source: North West London Five Year Forward View Vanguard bid

13. The process happening at the local level WSIC’s ten-step methodology for developing whole systems integrated care Source: WSIC Integrated Care Toolkit

14. Our research questions We explored: The way in which the WSIC programme is being designed Its involvement of local stakeholders in the processes of design The development and early implementation of early adopter schemes The extent to which the WSIC programme appears to be on track towards its objectives.

15. Our approach: pan-NWL level Formative evaluation Regular meetings with programme team Workshops with Early Adopters and programme team Focus group with Early Adopters Evaluation steering committee Interviews with senior stakeholders (~88) Observations of programme team meetings (120 hours) Survey of GPs in NWL (39% response rate, n=160)

16. Our approach: local level Four case study Early Adopters Interviews with key individuals Observations of Steering Committee meetings Survey with all Steering Committee members (60% response rate, n=109) Workshops with representatives from all Early Adopters

17. How do you evaluate such a large and complex programme? Combine breadth and depth In-depth interviews with limited number of people Workshops and focus groups Survey with all Steering Committee members Purpose: thoughts on progress, challenges, achievements Survey with GPs in NWL Purpose: contextual information to explore how easy or difficult it was going to be to roll out the programme ‘What would help you to deliver more integrated care?’ ‘Have you heard about the programme?’ Observations and document reviews (governance papers) Does what people say match what is actually happening?

18. In such a multi-level programme, which level do you investigate? Identify where change is taking place (in integrated care projects – macro, meso, micro) Monitoring of pan-NWL issues (context: financial, provider landscape, commissioning; developments and progress with programme team) Monitoring of all Early Adopters Case study approach for in-depth exploration of change at the Early Adopter level

19. In such a multi-level programme, which level do you investigate? (2) How we selected the case studies: Where is the early adopter based? Geography, complexity of health economy, contextual differences. Who is involved in the early adopter? Target population characteristics, partners delivering the intervention, readiness for implementation. How will the early adopter be delivered? Scale of initiative in terms of quantity, scale of initiative in terms of quality, extent of social care involvement.

20. How do you evaluate something that is constantly changing? Benefits of a formative evaluation Benefits of a realist evaluation approach Interviews/surveys at different time points Repeat interviews with same individual Develop a key contact within the programme team Board minutes are an excellent resource Keep up to date with local and national developments

21. Methodological challenges How do you evaluate such a large and complex programme? Take advantage of different methodologies Observations In such a multi-level programme, which level do you investigate? Both organisational and local How do you evaluate something that is constantly changing? Timing of research Approach to evaluation Regular contact Tracking national and local policy context

22. www.nuffieldtrust.org.uk Sign-up for our newsletter www.nuffieldtrust.org.uk/newsletter Follow us on Twitter (http://twitter.com/NuffieldTrust) © Nuffield Trust