Presentation on theme: "Using Systems Thinking to Improve Safety in Radiation Therapy"— Presentation transcript:
1 Using Systems Thinking to Improve Safety in Radiation Therapy Prof. Nancy G. LevesonAeronautics and AstronauticsEngineering SystemsMIT
2 To understand and prevent accidents, must consider system as a whole And so these men of Hindustan Disputed loud and long, Each in his own opinion Exceeding stiff and strong, Though each was partly in the right And all were in the wrong.John Godfrey Saxe ( )Accidents causes are often oversimplified.Everyone see just part of the problem. The real answer is that all are right.Accidents always have many causal factors and a complete explanation has to include all of them.
3 Facts about Accidents Almost never have single causes “Root cause seduction”Accidents are complex processesUsually involve flaws inEngineered equipmentOperator behaviorManagement decision makingSafety cultureRegulatory oversightRoot cause seduction: makes us feel in control if we can reduce the cause to one or two things.If it is just the operator’s fault or the medical physicist’s fault, then we can fire him/her or retrain them or tell them to be more careful.All of these things interact to lead to losses.
4 Human Error as a CauseALL accidents are caused by “human error” (except “acts of God,” like hurricanes)Almost always there is:Operator “error”Flawed management decision makingFlaws in the physical design of equipmentSafety culture problemsRegulatory deficienciesEtc.Headlines: “human error”physical devices: designed by humans, why weren’t they designed to be fail safe or fault tolerant?That’s a human design or engineering error.
5 Do Operators Really Cause Most Accidents? When say human error, usually mean “operator error”Hindsight biasOperator error vs. design errorA euphemism.Do operators really cause most accidents?two factors here
6 Hindsight Bias “should have, could have, would have” (Sidney Dekker, 2009)“should have, could have, would have”
7 Overcoming Hindsight Bias Assume nobody comes to work to do a bad job.Assume were doing reasonable things given the complexities, dilemmas, tradeoffs, and uncertainty surrounding them.Simply finding and highlighting people’s mistakes explains nothing.Saying what did not do or what should have done does not explain why they did what they did.Investigation reports should explainWhy it made sense for people to do what they did rather than judging them for what they allegedly did wrong andWhat changes will reduce likelihood of happening again?7
8 Fumbling for his recline button Ted unwittingly instigates a disaster The second important factor is system design vs. operator errorAll human behavior is affected by the context in which it occurs.Fumbling for his recline button Tedunwittingly instigates a disaster
9 Operator Error: Traditional View Operator error is cause of most incidents and accidentsSo do something about operator involved (suspend, retrain, admonish)Or do something about operators in generalMarginalize them by putting in more automationRigidify their work by creating more rules and procedures
10 Operator Error: System View Operator error is a symptom, not a causeAll behavior affected by context (system) in which occursTo do something about error, must look at system in which people work:Design of equipment and interface with equipmentUsefulness of proceduresExistence of goal conflicts and production pressuresEtc.Morphine pumpCT scannersAn accident waiting to happenTubes – aircraft have miles of wiring that can be misconnected. So developed color coding and male/female connections and other things to prevent these problems.Why has it been so difficult for the medical world to adopt these same techniques. As I understand it, the manufacturers have fought this since the 90’s and theFDA has caved in.
12 Procedures Cannot guarantee safety Safety comes from people being skillful in judging when and how they apply.Old view: Safety improvements come from organizations telling people to follow procedures and enforcing this.New view: Safety improvements come from organizations monitoring and understanding the gap between procedures and practice.Checklists now a panacea.New view: MRSA infections in hospitalscan harange them about washing their hands or you can figure why they aren’t doing it
13 An Engineering View of Safety: Overview Safety is a control problemAccidents occur when the system design does not enforce constraints on safe behaviorSafety must be designed into a systemEngineering relies on modeling and analysis to analyze system designIdentify physical behavior that can lead to accidentsIdentify where human errors are prone to happenDesign or redesign the system to prevent accidentsSecond topic was: how to prevent them
14 Safety as a Control Problem Goal: Design an effective control structure that eliminates or reduces adverse events.Controls may be:Physical designProcessesSocial (cultural, policy, individual self-interest)[Need more than just checklists]Engineers use a proactive approachPredict and manage adverse effectsThrough modeling and analysis (identify scenarios that can lead to accidents)Proactive approach different than the typical healthcare approach of waiting for adverse events. While this may be necessary for the human body (since we have relatively poor models of how humans work), physical and social systems are designed and we do have very good models. No need to wait until an accident, collect statistics, and then fix the problem. Not only does this result in unnecessary loss of life or injury, it is not very effective. Once the system or machine has been designed (especially for machines), very hard to fix them to be safer. There is no excuse for most of the machine related errors we are finding, including errors caused by the users because the human-machine interface was so poorly designed.Best approach is to design these systems to be safe from the beginning. We have standard safety engineering techniques that are effective andhave been around for 50 years. You don’t hear about nuclear bombs accidentally detonating and, as another example, commercial airplane travelis remarkably safe. It’s just a matter of the medical device manufacturers improving their design techniques. And, because as I said earlier, accidentsare multifactorial, of designing controls also into all the parts of the system.
18 Every Controller Contains a Process Model Accidents occur when model of process is inconsistent with real state of process and controller provides inadequate control actionsModel ofProcessControlActionsFeedbackFeedback channels are critical-- Design-- OperationControlled Process
19 A Systems Engineering Approach to Radiation Therapy Safety Identify system hazardsEstablish system safety requirements to reduce the occurrence and/or consequences of hazardsEnsure they are enforced by or implemented in the safety control structureDo a hazard analysisIdentify unsafe control actionsIdentify the causes of the unsafe control actionsEliminate or control hazardsEstablish risk management controls and procedures
20 Identify Accidents and Hazards Injury or death of patient related to treatmentInjury or death of staff or visitorEquipment damagedEnvironmental damage?HazardH1. OverdoseH2. UnderdoseH3. Inadequate fractioningH4. Non-patient exposure to radiationH5. Equipment stress
21 Highest Level Safety Constraint (Requirement) “Process of care” must not be compromisedPatients, staff, and visitors must not be exposed to an unhealthy dose of radiationEquipment must not be stressed beyond documented design limits
22 RT Hazards and Safety Constraints H1: Patient tissues receive more dose than clinically desirableSC1: The system must be able to prevent delivery of higher than clinically desirable doseH2: Patient tumor receives less dose than clinically desirableSC2: The system must be able to deliver sufficient dose to treat the tumor.H3: Patient treatment is improperly fractionedSC3: Each fraction must not exceed more than TBD Gy and must not be delivered TBD’ hours after the previous one without treatment plan being reevaluated.H4: Non-patient (esp. personnel) is unnecessarily exposed to radiationSC4: The system must be able to prevent unnecessary exposure of personnel and non-patients to radiationH5: Equipment is subject to unnecessary stressSC5: The system must prevent excessive equipment exposure to radiation.
23 System Safety Requirements A complete, formally documented, effective, and safe clinical treatment plan must be created for each patient undergoing radiation treatment. A radiation oncologist must select and formally approve the plan ultimately selected for treatment.Changes to the treatment plan must be evaluated and approved by a radiation oncologist.Standard operating processes must be provided that have been evaluated for safety and effectiveness. If SOPs are tailored, they must be evaluated and approved byImmobilization treatment devices must provide accurate treatment delivery and must not restrict the treatment techniques.Radiation safety guidelines (ACR/ASTRO and NRC) must be followed when therapy uses unencapsulated radionuclides.Dosimetry treatment plan must administer intended dose of radiation to the target volume while minimizing radiation exposure to normal tissues.A pretreatment quality assurance program must be in place and followed for every patient. The QA program must provide for checking the accuracy of both the dose calculation and the data used for treatment.
24 System Safety Requirements Verification and documentation of accuracy of treatment delivery (conforms to original or latest clinical and dosimetric plans) must be provided (includes management of organ movement).Modification of initial treatment plan (to adjust for changes) must be approved by radiation oncologist.Equipment must be calibrated and maintained according to AAPM guidelines and applicable state and federal regulations concerning radiation treatment delivery technology.Procedures must be created and followed to ensure that any possible sign of impending machine malfunction is quickly recognized and diagnosed and any necessary corrective or reparative action is taken prior to use of the machine to deliver a clinical treatment to patient.All radioactive sources must be carefully controlled and monitored at least to the extent required by regulatory agencies.Radiation oncologist, along with other members of the team, must review and manage ongoing treatment to ensure that it is effective and safe.
25 System Safety Requirements Follow-up evaluation and care must be provided to manage acute and chronic morbidity resulting from treatment.A process must be established to monitor for unexpected morbidity, tumor relapse, [etc.], to identify any possible safety problems during treatment and to identify measures that might reduce the risk of toxicity for future patients. All suspicious findings must be thoroughly investigated and resolved.[Patients must receive an appropriate level of medical, emotional, and psychological care during and after treatment]Staff must be protected from accidental radiation exposure.Appropriate arrangements must be made for emergency patients.Procedures must be evaluated periodically to ensure they are being followed and, if not, then determine why. Use the information to improve the procedures.All emergency equipment and safety devices must be operational at all times during hazardous operations.
26 System Safety Requirements Management of change procedures must include hazard analysis for any planned change to individual treatment plans and to the facility itself including any safety-critical equipment.Procedures must be in place to identify and remediate any unplanned changes over time to behavior within the system or within its environment that can affect system hazards.Reporting systems must be created that follow Just Culture principles.Leadership must make all staff feel comfortable (and rewarded) for raising safety concerns without fear of reprimand or reprisal.All members of the team must be empowered to be active participants in improving the safety of clinical processes.Trends and migration toward states of higher risk must be identified and effective procedures created to disseminate this information to all staff and to provide corrective measures.
27 System Safety Requirements Procedures must be in place to identify and investigate thoroughly all serious or potentially serious incidents. Recommendations must be implemented to eliminate or mitigate all identified factors contributing to the adverse events. Follow-up must be provided to ensure that recommendations have been implemented and are effective. Lessons learned must be documented and disseminated.A process must be established to evaluate the safety (identify hazards) associated with any equipment purchased from vendors or created in the hospital. Thoroughness and quality of the vendor’s hazard analysis and design for safety must be a major criterion in selecting a vendor. A two-way communication channel must be established to provide on-going communication about errors, incidents, and potential hazards.
28 System Safety Requirements The hospital must establish the safety of integrated systems purchased from multiple vendors and the introduction of new equipment into the total clinical environment. Sophisticated hazard analysis methods must be used to identify potential safety concerns about individual equipment or the integrated equipment and operational environment.Hazard logs must be created and maintained and used in the investigation of adverse events and in periodic performance audits to ensure that hazards are being adequately controlled and that the staff is sufficiently educated about the hazards involved in their job. Leading indicators of increasing risk must be identified and monitored.All staff must be educated on the hazards of their job responsibilities and the equipment they operateHazard analysis must include the analysis of human–automation interaction. Design methods must be used to minimize any potential human errors and HMI hazards, including investigating and reducing the frequency of spurious alarms and providing error messages and indications of safe operating limits for any potentially hazardous operation.
29 System Safety Requirements Safety-related decisions must be independent from cost and efficiency concerns. Conflicts must be identified and transparent resolution procedures created and followed to resolve any conflicts.The hospital must have a documented safety policy and a documented safety management plan. This policy and management plan must be periodically reviewed and updated and communicated to staff. Conformance with the safety policy and safety management plan must be monitored.The hospital must create and maintain a comprehensive safety information system.Robust feedback channels must be provided to enhance risk awareness to those with responsibilities related to the safety of the patients and safety.
31 Trace System Safety Requirements to Safety Control Structure All personnel must have and maintain a minimum level of knowledge and training.[pointers to Board Certification Process, Education process including continuing education, responsible official for training at UCSD, each staff component][includes initial orientation, education, credentialing, continuing education, and periodic evaluation]Completion of any component of care must be appropriately documented in the patient recordPatients must be evaluated to determine if treatment is recommendedPatient evaluation must be conducted by a qualified physician in consultation with other team members.
37 Patient readiness (esp. position) no patient ready not ready Process variablePossible valuesCommentPersonnelclosenot close"close" is to be understood as "potentially leading to detrimental radiation exposure"."personnel" is to be understood as "non-patient" (can include visitors, family members...)"close" = close to beamline or inside the treatment room"not close" = not close to beamline and outside of the treatment roomPatient readiness (esp. position)no patientreadynot ready“Ready”: patient is in treatment room, at treatment point, in the correct position and ready for dose delivery"Not ready": patient is in treatment room, but not ready for dose delivery (e.g. incorrect position)Treatment plan IDrightwrongnone"Right"/"Wrong" refer to whether the correct treatment plan has been selected and loaded for the patient awaiting treatment."None": no treatment plan has been loadedEquipment readiness"Ready"/"Not ready": with respect to treatment startMastership statusmasternot master"Not master": other areas have the power to control beamline elementsFacility modetherapynon therapy"Therapy": facility is configured for patient treatment application. All the patient safety and machine interlocks are enabled and remote operator control is disabled."Non-therapy": facility is configured to allow more flexibility for experimental purposes. Some patient safety and machine protection interlocks are disabled by default. The interlocks can be remotely disabled/enabled by an operator.Treatment statusno treatmentin progressinterrupted"Interrupted": treatment was previously in progress, but was stopped before completion and is expected to resume for the dose delivery to be complete
38 Example: Operator Starting Treatment System hazards: …Controller: Area OperatorControl actions:Load steering fileStart treatmentSTPA Step 1: identify unsafe control actionsSTPA Step 2: identify unsafe scenarios that lead to the unsafe control actions
39 Example Unsafe Control Actions (1) Treatment is started while personnel are in room (↑H-R4)Treatment is started while patient is not ready to receive treatment (↑H-R1, H-R2Note: This includes “wrong patient position”, “patient feeling unwell”, etc.Treatment is started when there is no patient at the treatment point (↑H-R2, H-R3)Treatment is started with the wrong treatment plan (↑H-R1,H-R2)Treatment is started without a treatment plan having been loaded (↑H-R1,H-R2)Cite John Thomas here.
40 Example Unsafe Control Actions (2) Treatment is started while the beamline is not ready to receive the beam (↑H-R1, H-R5)Treatment is started while not having mastership (↑H-R1, H-R2, H-R4)Treatment is started while facility is in non-treatment mode (e.g. experiment or trouble shooting mode) (↑H-R1, H-R2)Treatment start command is issued after treatment has already started (↑H-R1, H-R2)Treatment start command is issued after treatment has been interrupted and without the interruption having adequately been recorded or accounted for (↑H-R1, H-R2)Treatment does not start while everything else is otherwise ready (↑H-R1, H-R2)Cite John Thomas here.
41 Hazard Causal Scenarios (Causes of Unsafe Control Actions) UCA4: Treatment is started with wrong treatment plan(missing input) – no treatment file available and TDS loads previously used one(wrong input) – error in treatment planning and treatment file is incorrect(wrong input) – operator loads file from previous fraction(distorted transmission) – changes to daily plan not correctly communicated/understood by operator(actuator failure) – GUI fails to transmit the new steering file and TDS uses previously loaded one….also: inadequate feedback (sensor failure, wrong sensor calibration, …), external perturbations etc.Why is this here? Didn’t you say that earlier? Too much detail and too much repetition.
42 Causal ScenariosScenario 1 - Operator was expecting patient to have been positioned, but table positioning was delayed compared to plan (e.g. because of delays in patient preparation or patient transfer to treatment area; because of unexpected delays in beam availability or technical issues being processed by other personnel without proper communication with the operator).Controls:Provide operator with direct visual feedback to the gantry coupling point, and require check that patient has been positioned before starting treatment (M1).Provide a physical interlock that prevents beam-on unless table positioned according to plan
43 Example Causal Scenarios (2) Scenario 2 - operator is asked to turn the beam on outside of a treatment sequence (e.g. because the design team wants to troubleshoot a problem) but inadvertently starts treatment and does not realize that the facility proceeds with reading the treatment plan.Controls:Reduce the likelihood that non-treatment activities have access to treatment related input by creating a non-treatment mode to be used for QA and experiments, during which facility does not read treatment plans that may have been previously been loaded (M2);Make procedures (including button design if pushing a button is what starts treatment) to start treatment sufficiently different from non-treatment beam on procedures that the confusion is unlikely.
44 Organizational Aspects of Risk Example so far focuses on physical levelAlso requirements and control responsibilities at management level to satisfy system safety requirementsCan identify unsafe control actions and causal scenarios at higher levels of the control structure (perform a risk analysis) and build in controls to prevent themBehavior and control structures change over timePrevent migration to higher levels of riskDetect when occurs