Understanding and Presenting Quality Data in Healthcare Center for Clinical Effectiveness Loyola University Health System

PDSA Plan Plan an intervention including a plan for collecting data Do Perform the intervention Study Analyze the data and study the results Act Refine the change based on what was learned How do you get the team to agree on one analysis? Plan Plan an intervention including a plan for collecting data Do Perform the intervention Study Analyze the data and study the results Act Refine the change based on what was learned

3 Why graph data? YearJanFebMarAprMayJunJulAugSepOctNovDec

4 What’s your analysis? Mortality before and after new protocol 4% 5% 0% 1% 2% 3% 4% 5% 6% Percent mortality New Protocol in January

5 More details, so what’s the story? Feb-04 Apr-04 Jun-04 Aug-04 Oct-04 Dec-04 Feb-05 Apr-05 Jun-05 Aug-05 Oct-05 Dec-05 Percent mortality Mortality before and after new protocol New Protocol introduced in January 7% 8% 2% 3% 4% 5% 6%

Variation “Extent to which a thing varies; amount of departure from a position or state; amount or rate of change” - Webster’s Collegiate Dictionary

7 Two types of variation Common Cause Variation… 1.Regular, random, or expected variation 2.Not “assignable” to any specific cause 3.A natural part of all processes 4.Performance remains predictable 5.No “real change” Special Cause Variation… 1.A real change (assignable variation) 2.Not an essential part of a process 3.The underlying cause should always be identified 4.Sometimes unanticipated What kind of variation was this? Common Cause Variation… 1.Regular, random, or expected variation 2.Not “assignable” to any specific cause 3.A natural part of all processes 4.Performance remains predictable 5.No “real change” Special Cause Variation… 1.A real change (assignable variation) 2.Not an essential part of a process 3.The underlying cause should always be identified 4.Sometimes unanticipated

8 How can I differentiate… – Natural variation from – Meaningful change So what? – Natural variation from – Meaningful change

9 Results of misinterpreting random variation as a significant change We try to “correct” random variation We celebrate random variation How can we avoid these mistakes?

Control Charts Getting the team on the right track

11 Parts of a control chart

This information is confidential and to be used for quality improvement purposes only Month Jan-03 Feb-03Mar-03 Apr-03 May-03 Jun-03 Jul-03 Aug-03 Sep-03 Oct-03 Nov-03 Dec Goal = 0.3 Title Rate Analysis: ???

Analysis: The rate remains consistently above goal.

This information is confidential and to be used for quality improvement purposes only Month Jan-03 Feb-03Mar-03 Apr-03 May-03 Jun-03 Jul-03 Aug-03 Sep-03 Oct-03 Nov-03 Dec-03 Jan-04 Feb-04Mar-04 Apr-04 May-04 Jun-04 Jul-04 Aug Goal = 0.3 Title Rate Analysis: ???

Analysis: The rate remains consistently above goal.

Analysis: ???

Analysis: The rate has decreased past goal (0.26 since December 2004). Intervention #1 was begun in Nov-04, and phase-in of intervention #2 began in Feb-05. Intervention #2 Intervention #1

18 Unwanted special causes VS

19 What do you do with unwanted special cause variation?

20 1.Identify and label the cause 2.Correct (if a problem)

Analysis: This process is unpredictable

Self Test

What is the mean? Is this process predictable or not predictable? Can you predict the future performance (range)? 91.5% 80% - 100% What is the mean? Is this process predictable or not predictable? Can you predict the future performance (range)?

What is the mean? Is this process predictable or not predictable? Can you predict the future performance (range)? -1.3 (combined pre and post intervention) Unclear (due to improvement) What is the mean? Is this process predictable or not predictable? Can you predict the future performance (range)? Percent Title This information is confidential and to be used for quality improvement purposes only. Quarter UCL = 2.3 Mean = -1.3 LCL = Q1 (N=130)2002 Q2 (N=133)2002 Q3 (N=117)2002 Q4 (N=111)2003 Q1 (N=121)2003 Q2 (N=102) 2003 Q3 (N=88) 2003 Q4 (N=118)2004 Q1 (N=117)2004 Q2 (N=130)2004 Q3 (N=112)2004 Q4 (N=130)2005 Q1 (N=152)2005 Q2 (N=137)2005 Q3 (N=130)2005 Q4 (N=155)2006 Q1 (N=140) Intervention

What is the mean? Is this process predictable or not predictable? Can you predict the future performance (range)? 80.8% Unsure (unpredictable) What is the mean? Is this process predictable or not predictable? Can you predict the future performance (range)?

What is the mean? Is this process predictable or not predictable? Can you predict the future performance (range)? Unclear (due to improvement) 143 Can separate before after intervention What is the mean? Is this process predictable or not predictable? Can you predict the future performance (range)? Value Title These data are confidential and to be used for quality improvement purposes only. Month (number of results) UCL = Mean = 143 LCL = /2004 n=(11031)05/2004 n=(12789) 06/2004 n=(6269)07/2004 n=(7022) 08/2004 n=(12089)09/2004 n=(12990)10/2004 n=(12739)11/2004 n=(12563)12/2004 n=(13853)01/2005 n=(13486)02/2005 n=(10513)03/2005 n=(13535)04/2005 n=(10912)05/2005 n=(11773)06/2005 n=(10645)07/2005 n=(10543)08/2005 n=(13798)09/2005 n=(12604)10/2005 n=(11875)11/2005 n=(13137)12/2005 n=(15766)01/2006 n=(14068)02/2006 n=(12611)03/2006 n=(14501) Intervention

What is the mean? Is this process predictable or not predictable? Can you predict the future performance (range)? 0.1/100 cases Neither (too early) Unsure (too early) What is the mean? Is this process predictable or not predictable? Can you predict the future performance (range)?

What is the mean? Is this process predictable or not predictable? Can you predict the future performance (range)? 0.51/100 days 0–1.5 /100 days What is the mean? Is this process predictable or not predictable? Can you predict the future performance (range)?

What is the mean? Is this process predictable or not predictable? Can you predict the future performance (range)? 0.3% 0-1.7% What is the mean? Is this process predictable or not predictable? Can you predict the future performance (range)?

30 Benneyan RG, Lloyd RC, Plsek PE. Statistical process control as a tool for research and healthcare improvement. Qual Saf Health Care 2003; 12: Carey, Raymond G. and Lloyd, Robert C. Measuring Quality Improvement in Healthcare: A Guide to Statistical Process Control Applications. White Plains, NY: Quality Resources Press, Wheeler Donald J, Understanding Variation: The Key to Managing Chaos. Knoxville, TN: SPC Press, Additional SPC Reading: