IAEA E-learning Program Safety and Quality in Radiotherapy

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Sections: 8.1 Purpose and principles 8.2 Treatment planning 8.3 Information transfer 8.4 Calibration Here’s a reminder of the outline of this Module.

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Sections: 8.1 Purpose and principles 8.2 Treatment planning 8.3 Information transfer 8.4 Calibration In this section, we will perform a partial Failure Modes and Effects Analysis on the process of calibrating a treatment machine.

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Section 4: Calibration OBJECTIVES To work through a failure modes and effects analysis using postulated failures that could arise during treatment machine calibration. Here is the section objective; to work through a failure modes and effects analysis using postulated failures that could arise during treatment machine calibration

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Section 4: Calibration The Toulouse Incident Reported 2007 at Hôpital de Rangueil in Toulouse, France. In April 2006, the physicist in the clinic commissioned the new BrainLAB Novalis stereotactic treatment unit. This unit can operate with microMLC’s (3 mm leaf-width) or conical standard collimators. A brief reminder of the incident we looked at in Module 2. This is an example of what can go wrong during machine calibration. However, we will look at the more general situation of calibrating a treatment machine rather than looking at this specific incident. IAEA Prevention of accidental exposure in radiotherapy

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Section 4: Calibration Potential Failure Modes Let’s consider a few of the things “that could go wrong during a machine calibration”. An inappropriate calibration device is used. A 3% error in pressure is applied. The wrong source chamber distance, by 1.5 cm, is set. We start by postulating a few Potential Failure Modes. One, an inappropriate calibration device may be used. Two, a 3% error in pressure could be applied or three, the wrong source chamber distance, by 1.5 cm, might be set. These examples include using a chamber that is too large for measuring small fields as happened in Toulouse and elsewhere; misreading the barometer, and setting the chamber at the Source Axis Distance (SAD) plus dmax instead of SAD. Of course, we could include similar Failure Modes but with larger discrepancies such as a 10% error in atmospheric pressure. However, these examples will suffice for our present purposes.

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Section 4: Calibration A High Level Process Map1 These Failure Modes are clearly associated with Equipment and Software quality management

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Section 4: Calibration Potential Failure Modes We can enter these into the worksheet. We can enter these potential failure modes into the Worksheet

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Section 4: Calibration Potential Failure Modes Abbreviated to fit on the full worksheet. We abbreviate the descriptions to fit into the worksheet. SCD stands for Source-Chamber Distance.

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Section 4: Calibration Severity Now we ask “how bad would it be if these things happen?” An inappropriate calibration device is used. A 3% error in pressure is applied. The wrong source chamber distance, by 1.5 cm, is set. Now we ask “how bad would it be if these things happen?” for each failure mode.

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Section 4: Calibration Severity And we assign a number between 1 and 10 to describe the severity of the failure mode. An inappropriate calibration device is used. 10 A 3% error in pressure is applied. 4 The wrong source chamber distance, by 1.5 cm, is set. 4 And we assign a number between 1 and 10 to describe the Severity of the Failure Mode. Of course the Severity will depend on how bad the error is. In the case of the inappropriate measuring device we know this can be really bad from the small field calibration errors in Toulouse and elsewhere. We’ll give this a Severity of 10. A 3% error in the atmospheric pressure will result, in the absence of other errors, in  3% error in dose. As this is barely beyond the usual tolerance we apply to routine output checks of 2%, we’ll give this a Severity of 4. The wrong SCD by 1.5 cm for a 100 cm Source Axis Distance machine is also 3%. We’ll give this a Severity of 4 too. It is important to note that errors at Commissioning typically affect multiple patients and hence can have high Severity values. If the second and third Failure Modes we have identified here only applied to one fraction on one patient we could adopt lower Severity values, perhaps 1 for each.  in It is important to note that errors at commissioning typically affect multiple patients and hence can have high severity values.

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Section 4: Calibration Severity We enter our severity scores into our worksheet. We enter our Severity scores into our Worksheet

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Section 4: Calibration Failure Pathways Now we ask “how could these things happen?” A 3% error in pressure is applied. The barometer is not properly calibrated. A transcription error. The temperature-pressure calculation is incorrectly performed. Now we ask “how could these things happen?” We’ll pick one of those Failure Modes and speculate about how such a Failure could occur. If we pick “A 3% error in pressure is applied’ then possibilities include the barometer not being properly calibrated, a transcription error and the temperature-pressure calculation being incorrectly performed.

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Section 4: Calibration Failure Pathways We enter these into the worksheet. We enter these into our Worksheet Again, we’ve had to shorten the descriptions to make them fit into this illustration.

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Section 4: Calibration Occurrence We then ask “how likely are these things to happen on a scale of 1 to 10?” The barometer is not properly calibrated. 2 A transcription error. 3 The temperature-pressure calculation is incorrectly performed. 4 We then ask “how likely are these things to happen on a scale of 1 to 10?” Let’s assume the barometer is calibrated at a standards laboratory every 2 years. In that case the chance that is not properly calibrated now depends on the likelihood of drift which we estimate, with an Occurrence value of 2, to be low. Transcription errors are not infrequent. We’ll give this failure pathway a Severity of 3. They are more likely if there are distractions in the neighbourhood which is common. Distractions and interruptions can also make calculation errors more likely. Let’s give this a Severity of 4.

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Section 4: Calibration Occurrence We enter these into the worksheet. We enter these into our table

Detectability Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Detectability Section 4: Calibration Occurrence We then ask “how likely is it that these pathways will not be interrupted on a scale of 1 to 10?” The barometer is not properly calibrated. 10 A transcription error. The temperature-pressure calculation is incorrectly performed. We then ask “how likely is it that these pathways will not be interrupted on a scale of 1 to 10?” On the assumption that there is no second check on the calibration of this machine there is a very high probability that theses Failure Pathways will be fully open and that patients will be treated with doses that are in error by 3%. However, if there were to be a second check on the calibration, the Detectabilities for the second and the third Failure Pathways could be reduced perhaps down to 2. This is exactly why we do second checks. However, it would be hard to identify any drift in the barometer with the type of 2nd checks we normally do so the Severity for the first Failure Pathway would not drop much with a second check. Just to repeat, the high Detectability values used here, meaning these failure pathways are very unlikely to be intercepted, are based on the assumption that there is no second checking.

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Section 4: Calibration Detectability We enter these too. We enter these too

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Section 4: Calibration Risk Priority Numbers Finally we enter the risk priority numbers into our worksheet. Finally we calculate and enter the Risk Priority Numbers into our Worksheet Here are our final numbers. The numbers that you will come up with through your own FMEA will reflect your local practice.

Numbers are for illustration only. MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Section 4: Calibration Risk Priority Numbers Numbers are for illustration only. Just to emphasize again. The numbers in our table are for illustration only.

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Section 4: Calibration Risk Priority Numbers Now we can prioritize process and quality management changes starting with the highest risk priority number and working down. Failure Pathway Risk Priority Number The temperature-pressure calculation is incorrectly performed. 160 A transcription error. 120 The barometer is not properly calibrated. 80 Now we can prioritize process and quality management changes starting with the highest risk priority number and working down. This analysis suggests that the riskiest process is the calculation of the temperature pressure correction. So, this is the activity to which we would first direct our attention. However, close behind is the possibility of a transcription error compromising the quality and safety of a treatment. From this analysis it appears that the risk of the barometer not being properly calibrated is the lowest of the three considered. These 3 RPNs have been estimated in the absence of an appropriate second check on the machine calibration.

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Section 4: Calibration A Cautionary Note The probabilities of occurrence and detectability are critically dependent on the processes involved. For this reason, adopting occurrence and detectability values from the experience of other clinics is not only not valid but could be quite misleading. We have assumed no second check on the machine calibration. Considering the detectability values we have used here we can quickly appreciate the benefit of a second check. For equipment which is not familiar to the physics staff, an external, independent confirmation of the calibration is the safest strategy. We’ve seen this before but it’s worth repeating. The probabilities of Occurrence and Detectability are critically dependent on the processes involved. For this reason, adopting Occurrence or  Detectability values from the experience of other clinics is not only not valid but could be quite misleading. We’ve assumed no second check on the machine calibration. Considering the Detectability values we have used here we can quickly appreciate the benefit of a second check. For equipment which is not familiar to the physics staff, an external, independent confirmation of the calibration is the safest strategy.

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Section 4: Calibration summary We have: We have worked through a failure modes and effects analysis using postulated failures that could arise during treatment machine calibration. Here is what we’ve done in this section. We have worked through a Failure Modes and Effects Analysis using postulated Failures that could arise during treatment machine calibration.

Safety and Quality in Radiotherapy MODULE 8: failure modes and effects analysis Safety and Quality in Radiotherapy Section 4: Calibration References and additional resources E Ford, L Fong de los Santos, T Pawlicki, et al.. Consensus recommendations for incident learning database structures in radiation oncology. Medical Physics 39 (2012) 7272-7290. https://www.ncbi.nlm.nih.gov/pubmed/23231278