1. Quality Improvement Methodology – Next Steps

2. Purpose of this Session
Consider the components of a learning system in your own improvement activity :
4. Sequential testing of new theories
6. Planning for spread at scale
What stage are you at in relation to these aspects of the improvement journey?
What do you need to do next to ensure that your tests are scaled up and spread correctly?
We are going to look more closely at a couple of components of the learning system to help you think about where you are currently at and what you could do next in order to take your current work to this stage.

3. Aim Measures Changes Testing & Implementation
Hopefully you have opted for this session because you are feeling comfortable with the MFI and how it can be or is being applied in practice to take forward improvement activity in your CPP or within your own practice.
Reinforce the 3 questions and PDSA cycle to plan and guide improvement activity…
The Improvement Guide, API 3 3

4. Cycles of Tests Build Confidence
Northwest Improvement Initiative
Cycles of Tests Build Confidence
Changes that will result in improvement
Learning from data
Proposals, theories, hunches, intuition
You should also be familiar with the concept of multiple sequential cycles to test your theory by gradually building your learning through trying out a theory in different conditions with different people before it is ready to be implemented into everyday practice.
Working our way up to implementation is a matter of building belief in the effectiveness of a change.
First tests are small scale, under optimal conditions: changes that fail here are discarded quickly.
If they succeed, we explore them in more detail: by replication under other circumstances, with different practitioners, in different locales, etc. If the change looks robust, it will gather the approval it will need for implementation. Along the way, we’re building confidence in the change.
It also gives us the opportunity to make adaptations as we test so that the end product is fit for purpose across all the areas or population you need it to be.
© R. Scoville & I.H.I. 31

5. Start small 1 child 1 day 1 family 1 setting
Move to 3,5,7…. as confidence grows
As you know – beginning with a very small test population allows you to test something out without making a system change that you are not convinced is going to add value….
But it is the activity of scaling up the testing so that you grow confidence in the new process or activity through learning and adapting as you go that really builds the momentum for making improvement happen across the wider population.

6. You can only learn as quickly as you test!
Northwest Improvement Initiative
You can only learn as quickly as you test!
Years
Quarters
Months
Weeks
Days
Hours
Minutes
Drop down next “two levels” to plan test cycle!
And if you have been involved in carrying out tests - you will know that you only learn as quickly as you test….
And the more regular the PDSA tests are carried out the quicker you become knowledgeable about the practical application of the theory you are working on.
© R. Scoville & I.H.I.

7. Components of a Learning System
System level measures
Explicit theory or rationale for system changes
Segmentation of the population
Learn by testing changes sequentially
Use informative cases: “Act for the individual learn for the population”
Learning during scale-up and spread with a production plan to go to scale
Periodic review
People to manage and oversee the learning system
From Tom Nolan PhD, IHI
This is a key component of a learning system, which engages a number of people at the point of service delivery in improvement activity allowing the opportunity to influence and shape ‘how’ the theory can best be applied in practice. 7

8. Sequential Testing & Scale Up
Aim
Achieve improved communication process for HV and SW handovers using standardised format
Data/learning/adapting
Implement
Cycle 6: Test all handovers-1 week
Cycle 5: other team HVs/SWs- 1 day
Cycle 4:Test with all handovers-1 HV/1week
Cycle 3: Test with 5 more families/ same HV (Mon/Tu)
Hunch/Theory
A structured handover will ensure accurate information sharing between prof teams
Cycle 2: Test with 3 family handovers/ same HV (Wed)
Cycle 1: Test developed handover form 1 HV/1 SW 1 family (Mon)

9. Learning Through Sequential Testing at your tables discussion time
Where are you currently? Can you describe an example of the following? Multiple tests with different people/under different conditions? Learning/data captured to describe your testing journey?
Thinking about your current activity – consider these questions……
Take 5 mins to discuss with your neighbour
If you don’t have a worked example at the moment – can you describe a potentially high impact theory that you may be able to influence in the future?

10. Testing v Implementation
Testing – Trying and adapting existing knowledge on small scale. Learning what works in your system. Multiple tests in a variety of conditions…………………
Implementation – Making the change a part of the day-to- day operation of the system. Permanent change
Would the change persist even if its champion were to leave the organisation?
Avoid implementation until confident that processes are robust
Reminder that this will require process measurement to inform and assure of the reliability and sustainability of the change as it is imbedded in practice….

11. How Many Tests?

12. Components of a Learning System
System level measures
Explicit theory or rationale for system changes
Segmentation of the population
Learn by testing changes sequentially
Use informative cases: “Act for the individual learn for the population”
Learning during scale-up and spread with a production plan to go to scale
Periodic review
People to manage and oversee the learning system
From Tom Nolan PhD, IHI
The next crucial step when you have carried out multiple tests and are beginning to scale up the testing is to consider a plan that will enable you to take this work to scale across the wider community or population. 12

13. Measurement and Feedback Communication (awareness & technical)
A Framework for Spread
Leadership
Topic is a key strategic initiative
Goals and incentives aligned
Executive sponsor assigned
Day-to-day managers identified
Better Ideas
Develop the case
Describe the ideas
Set-up
Target population
Adopter audiences
Successful sites
Key partners
Initial spread plan
Knowledge Management
Measurement and Feedback
Communication (awareness & technical)
Social System
Key messages
Communities
Technical support
Transition issues
Institute for Healthcare Improvement

14. Things to Consider Checklists For Spread
Leadership, Better Ideas & Set Up
General Communication & knowledge
transfer
eveloping Measurement, Feedback
and Knowledge Management Systems

15. Take a strategic approach to scaling up
Level of testing
Strategy 1 5 25
125
250

16. Increasing uptake of Healthy Start Vitamins
Level of testing
Strategy 1
1 EY practitioner gives vits to 1 mum 5 25
125
250
Free vitamins in Asda with every pregnancy kit
This is a wild example but we need something shocking to make them realise how different strategy between 1 scale and 250 scale – will obviously need to consider staffing and other resources – I think giving them away free with all pregnancy kits is a great idea by the way!!

17. Are You Ready for Scale up & Spread? Table discussion time
In the context of your current improvement activity: Have you been testing a theory so that it could be considered ready for implementation in the area you are working? and/or… Do you have a strategy for moving to scale up and spread to other sites/teams?
Breakout discussion at tables……
Have a think about where you have got to on this journey…. At what stage is your current activity and improvement work?
Continuing to test – not ready to scale up?
Scaled up in own team/office – but not yet considered spread to wider system?
Scaled up and spread across the system? If so, how did you do this….? Do you have data to support your improvement?

18. The Seven Spreadly Sins
Step 1 Start with large pilots
Step 2 Find one person willing to do it all
Step Expect vigilance and hard work to solve the problem
Step If a pilot works then spread the pilot unchanged
Step Require the person and team who drove the pilot to be responsible for system-wide spread
Step Look at process and outcome measures on a quarterly basis
Step Early on expect marked improvement in outcomes without attention to process reliability
To be clear DO NOT DO THIS AT HOME!!!
If you follow these steps your spread effort will fail…………
Institute for Healthcare Improvement

19. Sustaining Improvement
Having the correct measures to provide assurance that new processes are reliable Measuring compliance or satisfaction through regular and random sampling of the population Understanding the variation that exists in your data
In relation to measurement – don’t over react to individual data points – seek to understand the influencing factors that impact on the data.
This all comes back to understanding what the normal or common cause variation is in the system…..

20. What measures?
Outcome measures – directly relates to the overall aim what is the result? how is the system performing?
Process measures – are the processes that contribute to the aim performing as planned?
Balancing measures – assessing from different dimensions unanticipated consequences, other factors influencing the outcome
Eg.
Workstream outcome measure for developmental milestones
Process measures for activites that are taken forward by nursery teams and HVs – the new processes tested and in place to measure compliance with the process.
Balancing – does this scaled up activity have an unintended consequence on the team involved – eg the length time it takes to do an activity or to complete a proforma, does this prevent other necessary work from taking place?

21. T N V R O A I A I
If we don’t understand the variation that lives in our data, we will be tempted to…
Deny the data as it doesn’t fit with our view of reality
See trends where there are none
Try to explain natural variation as special events
Blame and give credit to people for things over which they have no control
Distort the process that produced the data
Kill the messenger!

22. Common Cause Variation

23. Measurement of Improvement
Define measures that will measure the impact of the
Improvement work over time
They will guide your progress through and beyond testing to implementation and monitoring for continuous improvement.
Different ways of measuring e.g.,
Percent compliance with process
A count of correct attempts/number of attendances
Verbal feedback /surveys
Measures monitor and guide your progress through an improvement journey.
And you will need to define the measures that are going to be able to demonstrate the efforts of your improvement work
A number of different approaches can be taken;
It could be percentage compliance with something
A count of correct attempts at doing something
Or verbal feedback

24. Why a run chart and not just a graph or table?
Month
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
Nov
Measure 83 80 81 84 85 68 87 89 92 91
Run Chart
Prompt sticker used on all referrals
Testing screening tool with 3 Families
Prompt sticker tested in case referral
Median = 84%
Testing screening tool with 5 families
Testing screening tool with 1 family
Testing with all Families
Month
HC Data Guide 24

25. Non-Random Rules For Run Charts An astronomical data point
A Shift:
6 or more
A Trend:
5 or more
Too many or too few runs
An astronomical data point
Source: The Data Guide by L. Provost and S. Murray, Austin, Texas, February, 2007:

26. What is this data telling you?
Month
Data

27. What is this data telling you?
Astronomical
Point
Shift
Median = 5
Trend
Shift
Month
Median

28. Measurement Principles
Develop aims before measuring
Design measures around aims ‘How Good By When’
Be clear on your operational definitions
Establish a reliable baseline
Track progress over time using annotated run charts
Teams need measures to give them feedback that the changes they are making are resulting in improvement
Need to understand common cause and special cause variation to ensure we don’t over/under react to situations

29. Appreciation of a System Understanding Variation
Complex system of interaction between people, procedures and equipment
Success depends on integration, not performance of individual parts
Theory of Knowledge
Change is prediction of improvement based on knowledge of the system
Learning from theory, experience Operational definitions are the basis for improvement with PDSA cycles for learning
Psychology
Interaction of people with systems
Motivation & will of individuals & teams
Situation awareness/decision making
Managing stress and fatigue
Helps planning for change management
Improvement
You really need to understand your whole system so that the messiness of life doesn’t catch you out.
What might systems tell us – most systems result from a complex system of interaction between people procedures and equipment. Success depends on integration, not the performance of individual parts.
What might psychology tell us – how people interact with each other and the system. Helps with planning for change management. Motivation and will of individuals and teams
What might variation tell us - everything we measure varies and we make decisions based on interpretation of this variation. Eg was the improved performance this week the result of the change we made or just luck. Examining data over time….concept introduced by Shewhart which allows us to learn from the data story of what has been happening
What might theory of knowledge tell us – change is a prediction of improvement based on knowledge of a system. If the changes do not result in an improvement as predicted we need to identify what happened and use the understanding and learning from this to help refine the theory. This is the basis for a learning cycle and this is built into the PDSA cycle.
Understanding Variation
Variation is to be expected – everything we measure varies We make decisions based on interpretation
Data over time – data story of what has been happening
Aims & Values

30. What’s the Scope of Change?
Improvement Science in Action
What’s the Scope of Change?
System Targeted for Implementation (Defined by Aim)
Single-unit prototype: segments
As you move from pilot testing to implementation to spread, your population of interest will need to be adjusted.
Spread to Total System (Additional units, sites, organisations)
Planning for spread is crucial – even at an early stage in your improvement work, it is not too soon to be thinking about the scope of this change in your system and wider population.
Understand the difference between testing and implementation and don’t be tempted to spread too quickly to the wider system without a strategy to support it – this will give you the best chance of achieving success for the long term. 30

31. Thank You