1. Measurement for Improvement
27 October 2011
Measurement for Improvement
Insert name of presentation on Master Slide
Presenter: Mike Davidge

2. Measurement for improvement2
Measurement for improvement
“Measurement is for improvement not judgement.” D. Berwick
Don Berwick – CEO of the IHI in America.
Today is about using data for improvement, the majority of the NHS data that we collect is used for judgement and benchmarking ourselves against each other.

3. The measurement journey
Research
Improvement
The measurement journey
The rest of this talk is about the measurement journey. We’ll kick off with the type of measurement we want to use.
What sort of measurement
There are 3 types of measurement – which are we talking about? Let’s look at one we are very familiar with in our day to day work of managing our organisations – measurement for judgement. Talk through the 2 examples.
I make no apologies for using examples from the past. Good examples are like good jokes, the old ones are always the best!
Judgement
Measurement for Improvement

4. The 3 reasons for measurement
Source: Robert Lloyd IHI 2006 based on Solberg et all 1998
Why is measurement for improvement different?
What are the characteristics then of this type of measurement? Bring up list of aspects and go through them. Now introduce (click) those for judgement.
Let’s bring our clinical colleagues in now. They often work to a different measurement paradigm – measurement for research. Click to bring up characteristics.
Why is M4I different? [don’t bring up table yet, leave to summarise]. Measuring the right things in the right way. Let’s deal with the right way first.
Measurement for Improvement

5. The 3 reasons for measurement
Why is measurement for improvement different?
Source: Robert Lloyd IHI 2006
Measurement for Improvement

6. Measurement for improvement6
Measurement for improvement
It’s not audit
Mike Davidge
Don Berwick – CEO of the IHI in America.
Today is about using data for improvement, the majority of the NHS data that we collect is used for judgement and benchmarking ourselves against each other.

7. % medicines reconciled in a Medical Admissions Unit
Pharmacy included
Form printed
Letter from CDs
Form piloted

8. Model for Improvement
Remember this model from yesterday?

9. 7 Steps to measurement 1 Decide aim 2 Choose measures
3 Define measures
Slide 13 – 7 steps to measurement
Say that measurement doesn’t just happen. It requires a well defined process. The diagram shows the 7 steps necessary to get measurement to work for you.
Move to next slide
For reference
Step 1 – Decide your aim
Step 2 – Choose your measures
Step 3 – Confirm how to collect and display your data
Step 4 – Collect your baseline data
Step 5 – Analyse and present your data
Step 6 – Meet to decide what it is telling you
Step 7 – Repeat steps 4 to 6 each month or more frequently
7 Repeat steps 4-6
6 Review measures
4 Collect data
5 Analyse & present

10. Would you be able to describe your aim in a couple of sentences?
Step 1 Decide aim
1 Decide aim
2 Choose measures
3 Define measures
Slide 14 - Step 1: Decide your aim
They have just done this as part of the groupwork but also link to Vision session
7 Repeat steps 4-6
6 Review measures
4 Collect data
5 Analyse & present
Would you be able to describe your aim in a couple of sentences?

11. Step 2 Choose measures 2 Choose measures
1 Decide aim
2 Choose measures
(late starts/finishes)
Patient
Experience
& Outcomes
Safety &
Reliability
Efficiency &
Value
Leadership
& High
Performing
Teams
Clinical
Incidents
Pain score
% Theatre
Utilisation
Delays
Training &
Development
Staff
Survey
Cancellations
Turnaround
Time
Satisfaction
Readmissions
Staff absence
% Patients
Complication
free in
Recovery
Exceptions from ‘
time out ’
checklist
% correct
equipment
to hand
Average time
patient starved
turnover
2 Choose measures
There are two tools to help you choose measures
Process Maps
Driver Diagrams or cause and effect diagrams
3 Define measures
Slide 15 - Step 2: Choose your measures
Again they have done this as part of groupwork
7 Repeat steps 4-6
6 Review measures
4 Collect data
5 Analyse & present

12. Step 3 Define measures
1 Decide aim
2 Choose measures
An operational definition is a description, in quantifiable terms, of what to measure and the steps to follow to measure it consistently
Are we measuring the same thing?
3 Define measures
Slide 15 - Step 2: Choose your measures
Again they have done this as part of groupwork
7 Repeat steps 4-6
6 Review measures
4 Collect data
5 Analyse & present

13. Measure checklist 2 sides of A4 Page 1:
Why important?
Who owns?
Definitions?
Goals?
Do your thinking up front and avoid mistakes
Slide 21 – Define measure
Explain that a clear definition is essential so everyone knows what the measure is and how to interpret the results
To help you we have devised a measures checklist. It comes in two sections
Section 1 – Define your measure
Section 2 – Agree your measurement process
Talk them through section 1
It is only a guide and not meant to be comprehensive
Section on goals – do you want to set any?

14. Collect Decisions, decisions What - All patients or a sample?
1 Decide aim
Decisions, decisions
What - All patients or a sample?
Who – what role?
How – hospital system or audit?
When – Real time or retrospective?
2 Choose measures
3 Define measures
Slide 20 – Define measures (5 minutes for slides 20-22)
Explain they will need to look in detail at each measure to decide how and what to collect, what analysis is needed etc. This means planning steps 4 to 6 for each measure
7 Repeat steps 4-6
6 Review measures
4 Collect data
5 Analyse & present

15. Analyse
1 Decide aim
“The type of presentation you use has a crucial effect on how you react to data”
Summarise
Visualise
2 Choose measures
3 Define measures
Slide 20 – Define measures (5 minutes for slides 20-22)
Explain they will need to look in detail at each measure to decide how and what to collect, what analysis is needed etc. This means planning steps 4 to 6 for each measure
7 Repeat steps 4-6
6 Review measures
4 Collect data
5 Analyse & present

16. Complaints
Here’s some data, what would you do with it?

17. How might we report these numbers? And what conclusions might we draw?
Yearly total as numbers
Quarterly average as numbers
Quarterly average as bar chart

18. We have 2 quarterly data points - is this an improvement?
Higher is better

19. Are we assuming something like this?

20. But it could be like this ...

21. Or this ...

22. Or this!

23. As a run chart

24. You will use these 99% of the time
How am I doing?
Where’s the problem?

25. Groupwork: How will you analyse?
For the measure or measures you have just defined, complete the Analyse section of the checklist
Note any issues you will need to resolve outside the workshop
People you need to contact
Specialist help you might need

26. Review measures
It is a waste of time collecting and analysing your data if you don't take action on the results
1 Decide aim
2 Choose measures
3 Define measures
Slide 18 – Group discussion Report & review (10 minutes for slides 18-19)
Explain they are now going to decide where the review meeting(s) will happen
As a whole group, use a flip chart (the facilitator may want to scribe at this point) and discuss how they will use the measures information:
i) how it will be reviewed and communicated – e.g. in a theatre user group, newsletter, theatre display board
ii) when will you review these – for each of the methods identified in (i) , how frequent will you do this
Show an example of a review meeting as set out in slide 19.
7 Repeat steps 4-6
6 Review measures
4 Collect data
5 Analyse & present

27. 7 Steps to measurement
You may not get it right first time! You may need several attempts to get it right for you
1 Decide aim
2 Choose measures
3 Define measures
Slide 16 – an iterative process
Explain the PDSA symbol represents the several tests you might need to do to get it right
7 Repeat steps 4-6
6 Review measures
4 Collect data
5 Analyse & present