Quality Improvement Methodology – Next Steps

Purpose of this Session Consider the components of a learning system in your own improvement activity : 4. Sequential testing of new theories 6. Planning for spread at scale What stage are you at in relation to these aspects of the improvement journey? What do you need to do next to ensure that your tests are scaled up and spread correctly? We are going to look more closely at a couple of components of the learning system to help you think about where you are currently at and what you could do next in order to take your current work to this stage.

Aim Measures Changes Testing & Implementation Hopefully you have opted for this session because you are feeling comfortable with the MFI and how it can be or is being applied in practice to take forward improvement activity in your CPP or within your own practice. Reinforce the 3 questions and PDSA cycle to plan and guide improvement activity… The Improvement Guide, API 3 3

Cycles of Tests Build Confidence Northwest Improvement Initiative Cycles of Tests Build Confidence Changes that will result in improvement Learning from data Proposals, theories, hunches, intuition You should also be familiar with the concept of multiple sequential cycles to test your theory by gradually building your learning through trying out a theory in different conditions with different people before it is ready to be implemented into everyday practice. Working our way up to implementation is a matter of building belief in the effectiveness of a change. First tests are small scale, under optimal conditions: changes that fail here are discarded quickly. If they succeed, we explore them in more detail: by replication under other circumstances, with different practitioners, in different locales, etc. If the change looks robust, it will gather the approval it will need for implementation. Along the way, we’re building confidence in the change. It also gives us the opportunity to make adaptations as we test so that the end product is fit for purpose across all the areas or population you need it to be. © R. Scoville & I.H.I. 31

Start small 1 child 1 day 1 family 1 setting Move to 3,5,7…. as confidence grows As you know – beginning with a very small test population allows you to test something out without making a system change that you are not convinced is going to add value…. But it is the activity of scaling up the testing so that you grow confidence in the new process or activity through learning and adapting as you go that really builds the momentum for making improvement happen across the wider population.

You can only learn as quickly as you test! Northwest Improvement Initiative You can only learn as quickly as you test! Years Quarters Months Weeks Days Hours Minutes Drop down next “two levels” to plan test cycle! And if you have been involved in carrying out tests - you will know that you only learn as quickly as you test…. And the more regular the PDSA tests are carried out the quicker you become knowledgeable about the practical application of the theory you are working on. © R. Scoville & I.H.I.

Components of a Learning System System level measures Explicit theory or rationale for system changes Segmentation of the population Learn by testing changes sequentially Use informative cases: “Act for the individual learn for the population” Learning during scale-up and spread with a production plan to go to scale Periodic review People to manage and oversee the learning system From Tom Nolan PhD, IHI This is a key component of a learning system, which engages a number of people at the point of service delivery in improvement activity allowing the opportunity to influence and shape ‘how’ the theory can best be applied in practice. 7

Sequential Testing & Scale Up Aim Achieve improved communication process for HV and SW handovers using standardised format Data/learning/adapting Implement Cycle 6: Test all handovers-1 week Cycle 5: other team HVs/SWs- 1 day Cycle 4:Test with all handovers-1 HV/1week Cycle 3: Test with 5 more families/ same HV (Mon/Tu) Hunch/Theory A structured handover will ensure accurate information sharing between prof teams Cycle 2: Test with 3 family handovers/ same HV (Wed) Cycle 1: Test developed handover form 1 HV/1 SW 1 family (Mon)

Learning Through Sequential Testing at your tables discussion time Where are you currently? Can you describe an example of the following? Multiple tests with different people/under different conditions? Learning/data captured to describe your testing journey? Thinking about your current activity – consider these questions…… Take 5 mins to discuss with your neighbour If you don’t have a worked example at the moment – can you describe a potentially high impact theory that you may be able to influence in the future?

Testing v Implementation Testing – Trying and adapting existing knowledge on small scale. Learning what works in your system. Multiple tests in a variety of conditions………………… Implementation – Making the change a part of the day-to- day operation of the system. Permanent change Would the change persist even if its champion were to leave the organisation? Avoid implementation until confident that processes are robust Reminder that this will require process measurement to inform and assure of the reliability and sustainability of the change as it is imbedded in practice….

How Many Tests?

Components of a Learning System System level measures Explicit theory or rationale for system changes Segmentation of the population Learn by testing changes sequentially Use informative cases: “Act for the individual learn for the population” Learning during scale-up and spread with a production plan to go to scale Periodic review People to manage and oversee the learning system From Tom Nolan PhD, IHI The next crucial step when you have carried out multiple tests and are beginning to scale up the testing is to consider a plan that will enable you to take this work to scale across the wider community or population. 12

Measurement and Feedback Communication (awareness & technical) A Framework for Spread Leadership Topic is a key strategic initiative Goals and incentives aligned Executive sponsor assigned Day-to-day managers identified Better Ideas Develop the case Describe the ideas Set-up Target population Adopter audiences Successful sites Key partners Initial spread plan Knowledge Management Measurement and Feedback Communication (awareness & technical) Social System Key messages Communities Technical support Transition issues Institute for Healthcare Improvement

Things to Consider Checklists For Spread Leadership, Better Ideas & Set Up General Communication & knowledge transfer eveloping Measurement, Feedback and Knowledge Management Systems

Take a strategic approach to scaling up Level of testing Strategy 1 5 25 125 250

Increasing uptake of Healthy Start Vitamins Level of testing Strategy 1 1 EY practitioner gives vits to 1 mum 5 25 125 250 Free vitamins in Asda with every pregnancy kit This is a wild example but we need something shocking to make them realise how different strategy between 1 scale and 250 scale – will obviously need to consider staffing and other resources – I think giving them away free with all pregnancy kits is a great idea by the way!!

Are You Ready for Scale up & Spread? Table discussion time In the context of your current improvement activity: Have you been testing a theory so that it could be considered ready for implementation in the area you are working? and/or… Do you have a strategy for moving to scale up and spread to other sites/teams? Breakout discussion at tables…… Have a think about where you have got to on this journey…. At what stage is your current activity and improvement work? Continuing to test – not ready to scale up? Scaled up in own team/office – but not yet considered spread to wider system? Scaled up and spread across the system? If so, how did you do this….? Do you have data to support your improvement?

The Seven Spreadly Sins Step 1 Start with large pilots Step 2 Find one person willing to do it all Step 3 Expect vigilance and hard work to solve the problem Step 4 If a pilot works then spread the pilot unchanged Step 5 Require the person and team who drove the pilot to be responsible for system-wide spread Step 6 Look at process and outcome measures on a quarterly basis Step 7 Early on expect marked improvement in outcomes without attention to process reliability To be clear DO NOT DO THIS AT HOME!!! If you follow these steps your spread effort will fail………… Institute for Healthcare Improvement

Sustaining Improvement Having the correct measures to provide assurance that new processes are reliable Measuring compliance or satisfaction through regular and random sampling of the population Understanding the variation that exists in your data In relation to measurement – don’t over react to individual data points – seek to understand the influencing factors that impact on the data. This all comes back to understanding what the normal or common cause variation is in the system…..

What measures? Outcome measures – directly relates to the overall aim what is the result? how is the system performing? Process measures – are the processes that contribute to the aim performing as planned? Balancing measures – assessing from different dimensions unanticipated consequences, other factors influencing the outcome Eg. Workstream outcome measure for developmental milestones Process measures for activites that are taken forward by nursery teams and HVs – the new processes tested and in place to measure compliance with the process. Balancing – does this scaled up activity have an unintended consequence on the team involved – eg the length time it takes to do an activity or to complete a proforma, does this prevent other necessary work from taking place?

T N V R O A I A I If we don’t understand the variation that lives in our data, we will be tempted to… Deny the data as it doesn’t fit with our view of reality See trends where there are none Try to explain natural variation as special events Blame and give credit to people for things over which they have no control Distort the process that produced the data Kill the messenger!

Common Cause Variation

Measurement of Improvement Define measures that will measure the impact of the Improvement work over time They will guide your progress through and beyond testing to implementation and monitoring for continuous improvement. Different ways of measuring e.g., Percent compliance with process A count of correct attempts/number of attendances Verbal feedback /surveys Measures monitor and guide your progress through an improvement journey. And you will need to define the measures that are going to be able to demonstrate the efforts of your improvement work A number of different approaches can be taken; It could be percentage compliance with something A count of correct attempts at doing something Or verbal feedback

Why a run chart and not just a graph or table? Month Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Measure 83 80 81 84 85 68 87 89 92 91 Run Chart Prompt sticker used on all referrals Testing screening tool with 3 Families Prompt sticker tested in case referral Median = 84% Testing screening tool with 5 families Testing screening tool with 1 family Testing with all Families Month HC Data Guide 24

Non-Random Rules For Run Charts An astronomical data point A Shift: 6 or more A Trend: 5 or more Too many or too few runs An astronomical data point Source: The Data Guide by L. Provost and S. Murray, Austin, Texas, February, 2007:

What is this data telling you? Month Data 11 1 3 4 5 9 10 4 11 7 10 10 7 6 22 4 2 2 1 3 4

What is this data telling you? Astronomical Point Shift Median = 5 Trend Shift Month Median 1 1 2 2 3 3 4 4 4 4 5 6 7 7 9 10 10 10 11 11 22

Measurement Principles Develop aims before measuring Design measures around aims ‘How Good By When’ Be clear on your operational definitions Establish a reliable baseline Track progress over time using annotated run charts Teams need measures to give them feedback that the changes they are making are resulting in improvement Need to understand common cause and special cause variation to ensure we don’t over/under react to situations

Appreciation of a System Understanding Variation Complex system of interaction between people, procedures and equipment Success depends on integration, not performance of individual parts Theory of Knowledge Change is prediction of improvement based on knowledge of the system Learning from theory, experience Operational definitions are the basis for improvement with PDSA cycles for learning Psychology Interaction of people with systems Motivation & will of individuals & teams Situation awareness/decision making Managing stress and fatigue Helps planning for change management Improvement You really need to understand your whole system so that the messiness of life doesn’t catch you out. What might systems tell us – most systems result from a complex system of interaction between people procedures and equipment. Success depends on integration, not the performance of individual parts. What might psychology tell us – how people interact with each other and the system. Helps with planning for change management. Motivation and will of individuals and teams What might variation tell us - everything we measure varies and we make decisions based on interpretation of this variation. Eg was the improved performance this week the result of the change we made or just luck. Examining data over time….concept introduced by Shewhart which allows us to learn from the data story of what has been happening What might theory of knowledge tell us – change is a prediction of improvement based on knowledge of a system. If the changes do not result in an improvement as predicted we need to identify what happened and use the understanding and learning from this to help refine the theory. This is the basis for a learning cycle and this is built into the PDSA cycle. Understanding Variation Variation is to be expected – everything we measure varies We make decisions based on interpretation Data over time – data story of what has been happening Aims & Values

What’s the Scope of Change? Improvement Science in Action What’s the Scope of Change? System Targeted for Implementation (Defined by Aim) Single-unit prototype: segments As you move from pilot testing to implementation to spread, your population of interest will need to be adjusted. Spread to Total System (Additional units, sites, organisations) Planning for spread is crucial – even at an early stage in your improvement work, it is not too soon to be thinking about the scope of this change in your system and wider population. Understand the difference between testing and implementation and don’t be tempted to spread too quickly to the wider system without a strategy to support it – this will give you the best chance of achieving success for the long term. 30

Thank You