Advanced Measures Mike Davies, MD FACP Mark Murray and Associates

Where we’re going…. Compass: Examples of systemization Compass: Examples of systemization Analysis to answer questions Analysis to answer questions –Run –SPC –Queuing –Modeling

Dr Provider #1: “warm and compassionate with patients, infinite patience with some of the toughest customers”

Is There an Access Problem? Does this Provider have sufficient Access Availability? Yes for sure! Yes I think so I am not sure I don't think so Absolutely not! Quiz

Dr Provider #1: “warm and compassionate with patients, infinite patience with some of the toughest customers”

Does Provider #1 Have Adequate Access? –Provider #1 Third next available: >7 days in all months! –Provider #1 CUSS Past percentage appts utilized: >90% in 11 of 12 months (Dis) Continuity Measure < 10% (may depend upon facility/ Primary Care structure) (Dis) Continuity Measure < 10% (may depend upon facility/ Primary Care structure) –Provider #1 Continuity: 9% Diverted Demand to ER < 10% Diverted Demand to ER < 10% –Provider #1: 18%

Does Provider #1 Have Adequate Access? No!! No!! –Uniformly poor Access Availability throughout the entire year, with few available slots and a high third next available.

Diagnosing Access Problems Why doesn't this provider have adequate Access Availability? Not enough appointment slots for the panel size The provider is cancelling clinics too often The return visit rate is too high The missed clinic rate is too high Access utilization is suboptimal Not enough information presented in Dashboard to answer Live Meeting Poll

PCMM Panel of Patients Predicted #Slots Needed VISTA Profile: #Slots/Wk Estimated #Slots Available per Yr Doc # Supply/Demand Balance? Are there adequate slots per patient (Does estimated supply meet the predicted demand?) Are there adequate slots per patient (Does estimated supply meet the predicted demand?) –Compare Estimated #Slots Available with Predicted #Slots Needed –For Provider #1, these two measures are comparable, therefore this Provider is not under-slotted.

PCMM Panel of Patient s Predict ed #Slots Needed VISTA Profile: #Slots/ Wk Estimat ed #Slots Availab le per Yr CUSS Appt Slots in Past Year Deman d: Appts Schedul ed Past Year RVI>6 mo (>70%) Missed Clinic Rate (<10%) Doc # %15% Why Doesn’t Provider #1 Have Access?

Other ways to use the compass Relative comparisons Relative comparisons

144 days of data. Numbers are cumulative

3166 – 2362 = /1.64 = 490

144 days of data. Numbers are cumulative

When we don’t interpret variation correctly….. We see trends when there are none We see trends when there are none We explain natural variation as special events We explain natural variation as special events We blame or give credit when it’s undeserved We blame or give credit when it’s undeserved We don’t understand past performance or make accurate future predictions We don’t understand past performance or make accurate future predictions Ability to make improvements is limited Ability to make improvements is limited

Two Types of Variation Common Cause Common Cause –Inherent in current design of process –Predictable - stable –Due to “random chance” Special Cause Special Cause –Not inherent in process design – “unnatural” –Unpredictable – unstable –Due to explainable cause

Why Does Special and Common Cause Variation Matter? If uncontrolled variation (special cause variation)- identify special causes (may be good or bad) process is unstable process is unstable variation is extrinsic to process variation is extrinsic to process cause should be identified and “treated” cause should be identified and “treated” If controlled variation – (common cause variation) reduce variation, improve outcome process is stable process is stable variation is inherent to process variation is inherent to process therefore, process must be changed therefore, process must be changed

Can a Run Chart Detect Special Cause Variation? ---- YES! 1. Too many or too few runs 1. Too many or too few runs –One or more data points on the same side of the median –Do not include points ON the median 2. Shift: If more than 7-8 points in a run 2. Shift: If more than 7-8 points in a run 3. Trend: If more than 5-6 consecutive points up or down 3. Trend: If more than 5-6 consecutive points up or down 4. Stratification: See-saw pattern 4. Stratification: See-saw pattern

What is a Run? One or more consecutive data points on the same side of the median. One or more consecutive data points on the same side of the median. Do not include points ON the median in a run. Do not include points ON the median in a run.

Useful Observations Lower Limit Upper Limit

Summary of Key Points Become expert at creating run charts (it’s not that hard!) Become expert at creating run charts (it’s not that hard!) Use run charts to tell us if a change is an improvement Use run charts to tell us if a change is an improvement Use run charts to detect common and special causes of variation Use run charts to detect common and special causes of variation Post run charts widely so all can see the changes! Post run charts widely so all can see the changes!

Analyzing Variation – The MRI! Control Charts (or SPC charts) Control Charts (or SPC charts) –More sensitive than run charts Common/Special Cause Common/Special Cause –Define process capability –Allow predictions of process behavior –Can be easily created by simply analyzing the data in a run chart with more sensitive formulas

My trip to work Mean Upper process limit Lower process limit

How Do We Get a SPC Chart? Use individual values to calculate the Mean Use individual values to calculate the Mean Difference between 2 consecutive readings, always positive = Moving Range, mR Difference between 2 consecutive readings, always positive = Moving Range, mR Calculate the Mean mR Calculate the Mean mR One Sigma/standard deviation = (Mean mR)/d2* One Sigma/standard deviation = (Mean mR)/d2* –s or σ Upper Process Limit (UPL) = Mean + 3 s Upper Process Limit (UPL) = Mean + 3 s Lower Process limit (LPL) = Mean - 3 s Lower Process limit (LPL) = Mean - 3 s * The bias correction factor, d2 is a constant for given subgroups of size n (n = 2, d2 = 1.128) H.L. Harter, “Tables of Range and Studentized Range”, Annals of Mathematical Statistics, 1960.

SPC Formula Example

And that’s how you get one of these (A Control Chart)

Zone A Zone B Zone C Zone B Zone A 1 Sigma 2 Sigma 3 Sigma

X X X X X X X X X LCL UCL MEAN X X X X X X X X X X LCL UCL MEAN X Point above UCL Point below LCL Special causes - Rule 1

Special causes - Rule 2 2/3 Successive Points in Zone A on one side of the centre line LCL UCL

MEAN Seven points above center line Special causes - Rule 3 LCL UCL LCL UCL X X X X X X X X X X X X X X X X X X X X X Seven points below center line

MEAN Six points in a downward direction Special causes - Rule 4 LCL UCL LCL UCL X X X X X X X X X X X X X X X X X X X X X Six points in an upward direction

Special causes - Rule 5 X X X X X X X X X X X X X X X X X X X X Cyclic pattern X X X X X X X X X X X X X X X X X X X LCL UCL LCL UCL Trend pattern

Which Type of SPC Chart Should I Use? There are 30 or more types of SPC charts There are 30 or more types of SPC charts Which one we choose depends on the question we’re asking Which one we choose depends on the question we’re asking These are available on computers – no calculation needed These are available on computers – no calculation needed Most important thing is to choose the right chart for the right question….. Most important thing is to choose the right chart for the right question…..

Placeholder for Control Chart Demo

Pincher Creek Wait Data

How Much S to meet D? Common Cause Variation Common Cause Variation

Demand Min = 75 Max = 175

Supply needed is (90-50) = 72 Supply needed is (120-70) = 115

Supply Needed SN = Min (Max – Min) SN = Min (Max – Min) SN = (175 – 75) SN = (175 – 75) SN = SN = SN = 155 SN = 155 Note: Average = 125

If you know this: You can get this: Arrival Rate 50per hour Service Rate20per hourService Time Servers43minutes per car Queue Capacity5Effective Arrival Rate Utilization62% Traffic Intensity2.5 Avg Number of Cars in Queue0.394 Avg Number of Cars in System2.865 Avg Time in Queue0.008 Avg Time in System0.058 Probabilty of an Empty system7.51% Probabilty of having to wait30.66% Probabilty of a Full system1.17%

Queuing Allows Calculation of: Number of servers needed under various conditions (supply) Number of servers needed under various conditions (supply) Amount of wait resulting from a system Amount of wait resulting from a system ………..As long as the arrival rate is even, there are no unusual events, and the system is simple

Computer Modeling/Simulation Applications that mimic the behavior of real systems on a computer Applications that mimic the behavior of real systems on a computer Allows “playing” with the system Allows “playing” with the system Allows asking “what if” questions Allows asking “what if” questions Can see results of changes Can see results of changes