1. Quality Control and Patient Risk Curtis A. Parvin, Ph. D
Quality Control and Patient Risk Curtis A. Parvin, Ph.D. Manager of Advanced Statistical Research Quality Systems Division

2. Laboratory QC and Patient Risk
For the clinical laboratory patient risk is related to the reliability of the patient results they produce and report.
Laboratory Quality Control principles have been connected to the concept of patient risk for decades.
Specify the quality required of a patient result.
Determine the magnitude of “critical” out-of-control conditions with unacceptably high probability of producing patient results that fail to meet the required quality.
Design QC procedures with adequate power to detect “critical” out-of-control conditions.
2 |

3. Risk Management
A number of risk management guidelines have been published recently.
For manufacturers
ISO 14971: Medical devices – Application of risk management to medical devices
CLSI EP18: Risk management techniques to identify and control laboratory error sources
For the laboratory
ISO 22367: Medical laboratories – Reduction of error through risk management and continual improvement
CLSI EP23: Laboratory quality control based on risk management
3 |

4. The Risk Management Process
Risk Management provides a formal approach to
Identify potential failure modes in the lab
Rank the identified failure modes in terms of their risk
Establish policies and procedures to prevent or reduce (mitigate) the risks
Focus on the high ranked risks
This is generally a qualitative process
Risks are computed as a combination of
The probability of occurrence of harm
The severity of harm
4 |

5. Statistical QC and Risk Management
The question that we will address today is:
How do statistical QC design principles that address questions such as
How many QCs should I examine?
What QC rule(s) should I use?
How frequently should I schedule QC evaluations?
fit into the overall risk management process?
5 |

6. Risk Assessment (EP23 Figure)
Hazard Identification
Create a process map
Identify potential failures in each process step
Determine mechanisms in place to prevent or detect a failure
Risk Estimation
Assess the likelihood or probability of harm for each failure
Assess the severity of harm to a patient from each failure
Risk
Estimation
Is the residual risk
of harm clinically
acceptable
Risk Control
What control processes
are needed to lowerrisk
to an acceptable level No
Yes
The Laboratory’s QCP
Compile set of QC process into QCP
Review QCP for conformance to regulatory and accreditation requirements
Document and implement the set of control processes as the laboratory’s QCP
6 |

7. Risk Assessment (EP23 Figure)
Hazard Identification
Create a process map
Identify potential failures in each process step
Determine mechanisms in place to prevent or detect a failure
Risk Estimation
Assess the likelihood or probability of harm for each failure
Assess the severity of harm to a patient from each failure
Risk
Estimation
Is the residual risk
of harm clinically
acceptable
Risk Control
What control processes
are needed to lowerrisk
to an acceptable level No
Yes
The Laboratory’s QCP
Compile set of QC process into QCP
Review QCP for conformance to regulatory and accreditation requirements
Document and implement the set of control processes as the laboratory’s QCP
7 |

8. Probability of Harm
Sequence of Events Creating Risk of Harm for a Patient
Hazardous Situation P1 P2 P3 P4 P5 P6
Initiating
cause
Testing
process
failure
Incorrect
result
generated
Incorrect
result
reported
Misdiagnosis
Hazardous
medical
action
Patient
harmed
EP23-A, Figure 6
ISO 14971, Figure H.1
ISO/TS 22367, Figure B.1
Estimates of the probability of harm are desirable
For each potential mode of failure
Collectively for all possible failure modes
8 |

9. Estimating Probability of Harm
Initiating
cause
Testing
process
failure
Incorrect
result
generated
Incorrect
result
reported
Misdiagnosis
Hazardous
medical
action
Patient
harmed
P1: The probability that a testing process failure occurs
9 |

10. P1: Probability of a Failure Mode
P1 = how likely is a given failure mode going to occur
or more commonly
P1 can be the mean time between failures (MTBF)
Information regarding MTBF may come from
Manufacturer
Historical failure data
Literature
MTBF can be converted to mean number of patient results between failures (MPBF)
MTBF = 30 days
150 patient results examined per day (on average)
MPBF = 30 * 150 = 4500 patient results
10 |

11. Estimating Probability of Harm
MPBF
MPBF P1 P2 P2 P3 P4 P5 P6
Initiating
cause
Testing
process
failure
Incorrect
result
generated
Incorrect
result
reported
Misdiagnosis
Hazardous
medical
action
Patient
harmed
P2: The probability that a testing process failure generates incorrect patient results
11 |

12. P2: Probability of Producing an Incorrect (Unreliable) Result
The number of unreliable patient results produced during the existence of an out-of-control condition depends on
The quality required for patient results
The type and magnitude of the out-of-control condition
The power to detect the out-of-control condition
When (how often) QC evaluations are performed
An out-of-control condition may result in many unreliable patient results being produced
12 |

13. Unreliable Patient Results
ISO Clause 5.6.1: Laboratory QC should assure that patient results meet the quality required for their intended use
The quality of a patient result depends on the difference between the correct value and the value reported.
If the error in a patient’s result exceeds the allowable total error (TEa)
The result is considered unreliable (or incorrect).
It creates a hazardous situation for the patient.
13 | 13

14. Expected Number of Unreliable Patient Results Produced: E(NU)
The number of unreliable patient results produced during an out-of-control condition (red asterisks) will depend on
The magnitude of the out-of-control condition
The power of the QC rule – Detection
The number of patient specimens between QC events
E(Nu) = Expected number of unreliable patient results generated during an out-of-control condition
14 | 14

15. Estimating Probability of Harm
MPBF
E(Nu)
E(Nu) P2 P3 P3 P4 P5 P6
Initiating
cause
Testing
process
failure
Incorrect
result
generated
Incorrect
result
reported
Misdiagnosis
Hazardous
medical
action
Patient
harmed
P3: The probability that incorrect results that are generated will be reported
15 |

16. P3: Probability of Reporting an Unreliable Patient Result
The probability of reporting an unreliable result that leads to an incorrect action depends on
The number of unreliable results produced because of an out-of-control condition
How and when results are reported
The likelihood of identifying and correcting a reported unreliable result before an incorrect action is taken
16 |

17. Expected Number of Unreliable Patient Results Reported
E(Nu)
E(Nuf)
E(Nuc)
The patient results produced during an out-of-control condition can be divided into
Results prior to the last accepted QC event (Pre)
Results since the last accepted QC event (Post)
Bracketed QC: results aren’t reported
Immediate reporting: results should be repeated and updated in a timely fashion
E(Nuf) = Expected number of unreliable “final” results
E(Nuc) = Expected number of unreliable “correctable” results
17 | 17

18. Estimating Probability of Harm
E(Nuf)
MPBF
E(Nu)
E(Nuf) P3 P4 P4 P5 P5 P6 P6
Initiating
cause
Testing
process
failure
Incorrect
result
generated
Incorrect
result
reported
Misdiagnosis
Hazardous
medical
action
Patient
harmed
P4: The probability that an incorrect result leads to a misdiagnosis
P5: The probability that a misdiagnosis leads to a hazardous medical action
P6: The probability that a hazardous medical action leads to patient harm
P4*P5*P6: The probability than an incorrect result leads to patient harm
18 |

19. Probability of Patient Harm from an Unreliable Patient Result
The probabilities (P4, P5, P6) are associated with activities outside the laboratory
Their product equals the probability of patient harm given that an unreliable result has been reported
Ph|u = P4*P5*P6
Estimates of the probability of harm given an incorrect result are based on;
Medical literature
Medical judgment
Consultation with clinical colleagues
19 |

20. Estimating Probability of Harm
MPBF
E(Nu)
E(Nuf) P4 P5
Ph|u P6
Initiating
cause
Testing
process
failure
Incorrect
result
generated
Incorrect
result
reported
Misdiagnosis
Hazardous
medical
action
Patient
harmed
20 |

21. Estimating Probability of Harm: Risk Management + Statistical QC
MPBF
E(Nu)
E(Nuf)
Ph|u
Initiating
cause
Testing
process
failure
Incorrect
result
generated
Incorrect
result
reported
Patient
harmed
21 |

22. Harmed Patients from Failure Mode Occurrence
Ph|u*E(Nuf)
MPBF * * *
E(Nuf)
E(Nu)
E(Nuc)
Probability of patient harm is
The expected number of patients harmed due to incorrect results from a failure mode: Ph|u* E(Nuf)
Divided by the average number of patient results examined between failure mode occurrences: MPBF
Probability of Harm = Ph|u* E(Nuf) / MPBF
22 | 22

23. What Info Do We Need to Predict E(Nu) and E(Nuf)?
The quality requirement for the analyte
Allowable Total Error (TEa)
Test method performance
Instrument imprecision and bias
QC procedure
QC rule(s) used
Number of QC levels evaluated at each QC event
QC schedule (average number of patients examined for the analyte between QC events)
Needed to compute the
probability of an
unreliable patient result
The quality requirement for the analyte
Allowable Total Error (TEa)
Test method performance
Instrument imprecision and bias
QC procedure
QC rule(s) used
Number of QC levels evaluated at each QC event
QC schedule (average number of patients examined for the analyte between QC events)
The quality requirement for the analyte
Allowable Total Error (TEa)
Test method performance
Instrument imprecision and bias
QC procedure
QC rule(s) used
Number of QC levels evaluated at each QC event
QC schedule (average number of patients examined for the analyte between QC events)
Needed to compute the
expected # of
affected patient results
23 |

24. Probability of an Unreliable Result Due to a Systematic Error Condition
24 | 24

25. Probability of an Unreliable Result Depends on TEa, Method CV and Bias
25 | 25

26. Expected # Affected Depends on QC Rule and QC Schedule
26 |

27. Expected # of Unreliable Patient Results
27 |
27 |

28. Ways to Reduce (or Manage) Patient Risk
Estimate Risk
Reduce Risk
Rate of Occurrence of Failure Modes
Yes: manufacturer info, technical bulletins, product alerts
Some: lab environment/processes training
Very little: Medical devices
Number of Incorrect Results Reported
Yes: Statistical QC design & evaluation
A lot: Detection in analytical phase
A little: Detection in pre- and post- analytical phases
Probability of Harm from an Incorrect Result
Yes: Medical literature, medical judgment, consultation
Very little: Report formatting, education
28 |

29. Ways to Reduce (or Manage) Patient Risk
Manage the risk of unreliable patient results in the worst case
Limit the maximum values for E(Nuf) and E(Nuc)
Manage the overall expected risk of unreliable patient results
Limit the areas under the E(Nuf) and E(Nuc) curves
Manage the “defect rate” for unreliable patient results
Requires an estimate of the rate at which out-of-control conditions occur
29 |

30. Summary Risk management Statistical QC
The laboratory has two main mechanisms to estimate and reduce the risk of patient harm:
Identify as many potential failure modes as possible and seek ways to reduce occurrences of the identified failure modes
Risk management
Implement QC strategies that minimize the number of incorrect patient results that are reported when a failure mode does occur
Statistical QC
The combination enables a laboratory to address the full spectrum of patient risk implications for their operations
30 | 30

31. References
Parvin CA. Assessing the impact of the frequency of quality control testing on the quality of reported patient results. Clin Chem 2008;54:
Parvin CA, Yundt-Pacheco J, Williams M. Analytical assessment in the clinical laboratory: assessing analytical quality goals when the same analyte can be tested on multiple systems is explored. Adv Admin Lab 2011;20(1):28-31.
Parvin CA, Yundt-Pacheco J, Williams M. The focus of laboratory quality control: why QC strategies should be designed around the patient, not the instrument. Adv Admin Lab 2011;20(3):48-9.
Parvin CA, Yundt-Pacheco J, Williams M. Designing a quality control strategy: in the modern laboratory there are 3 questions that must be answered. Adv Admin Lab 2011;20(5):53-4.
Parvin CA, Yundt-Pacheco J, Williams M. The frequency of quality control testing: QC testing by time or number of patient specimens and the implications for patient risk are explored. Adv Admin Lab 2011;20(7):66-9.
Parvin CA, Yundt-Pacheco J, Williams M. Recovering from an out-of-control condition: the laboratory must assess the impact and have a corrective action strategy. Adv Admin Lab 2011;20(11):42-4.
Parvin CA, Yundt-Pacheco J, Williams M. Sigma metrics, total error budgets, and quality control: Make sure your test system performance and quality control procedures are aligned with your quality goals. Adv Admin Lab 2012;21(1):40-4.
31 |

32. 32 |

33. 33 |

34. THANK YOU!
34 | 34