1. New England Chapter of the AAPM 2013 Winter Meeting Newton, MA February 1, 2013 Making High Tech Radiotherapy Safe Howard Amols, Ph.D. Memorial Sloan Kettering Cancer Center New York, USA

2. Radiation Therapy Accidents

4. WHO World Alliance for Patient Safety –Literature review 1976-2007 –3125 patients affected by errors in radiotherapy –~1% resulted in death (middle and high income countries*) 98% of errors occurred in planning stage or during introduction of new systems or equipment –Of errors without adverse effects 55% planning 25% new systems or equipment 9% information transfer 10% treatment delivery –Mis-information or errors in data transfer- greatest bulk of “near misses” in modern radiotherapy services Radiation Therapy safety problems

5. 3 Myths 1.Radiation therapy accidents are rare 2.Most accidents happened long ago in the developing world 3.You need new high technology to have an accident

6. There’s been a focus on high-technology as the source of RT accidents. This is only partially true. New technology is often a contributing factor, but most RT disasters involving new technology required significant help from humans in order to escalate. Really big errors are a team effort! (more than one person has to make a mistake)

7. A Major Accident Requires: Equipment malfunction AND/OR an individual act of negligence AND a deficient QA/QC program, bad communications, inadequate training, lack of teamwork

8. If commercial airlines had the same error rates as hospitals there would be a major airline crash EVERY DAY. There is something fundamentally wrong with the historical method by which the medical profession has approached QA/QC:

9. “Those who don't know history are destined to repeat it.” Edmund Burke

10. Radiation Accidents: Common Threads new equipment + new software, but no new QA/QC understaffed, overworked, undertrained, rushed no internal redundancy, no external audits no common sense, no time outs bad communication, no central reporting too much faith in manufacturer design manufacturer and institutional denial unusual clinical results ignored by physicians

11. A recent IAEA analysis of reported treatment errors noted common threads: 1. Possibility of systematic software error never considered; 2. Manufacturer failed to inform users of previous incidents; 3. Error messages from system ignored or overridden; 5. Used software in unanticipated sequence that confused the system; 6. Input wrong data, misunderstood input requirements; 7. No physical dose measurements made to confirm; 8. New or recently upgraded equipment;

12. Life is more complicated than it used to be! Control software for Varian Truebeam Linacr (for example) has > 1 million lines of computer code! No single person can completely understand how such a system works No government regulatory agency has the expertise to review such a system Teamwork and communication is essential

13. RT is more complicated than it used to be! Manufacturers software engineers rarely have clinical experience and do not know how a clinic really operates People will always find new ways to make mistakes that are difficult to anticipated Quality Assurance must be an evolving process, to be reviewed and modified whenever new technology is implemented and when errors are discovered Study near-misses as well as actual errors Never underestimate the ingenuity of a fool, or someone who is being rushed!

14. Special Dangers of Hi-Technology 1.Systematic errors harder to detect 2.Humans get complacent. Don’t really check computers (evolution of R/V systems) 3.Many treatment components too complex for humans to check (e.g., DMLC files, IMRT files) 4. Many treatment aids/devices are invisible (dynamic wedge, MLC, etc.) 5. Errors made on day 1 can propagate 6. Programmers don’t understand what we do 7. We don’t understanding what programmers do 7. Too easy to `over ride’ warning messages 8. Manufacturers training programs often inadequate

15. My `favorite’ IMRT accident Almost everything that can go wrong in a radiation therapy accident occurred here! Best teaching tool I know of on the subject of radiation therapy errors

17. Background, IMRT accident March 2005, New York City A patient is to be treated with IMRT for head and neck cancer (oropharynx)

18. March 4 – 7: An IMRT plan is prepared: “1Oropharyn”. Verification plan created by TPS. EPID dosimetry confirms correctness. March 8: patient treated correctly with “1Oropharyn”. March 9-11: Fractions #2, 3 and 4 also correct. Verification images for the kV imaging system are created and added to the plan, now called “1AOropharyn”.

19. March 11: P hysician wants modified dose distribution (reducing dose to teeth) “1AOropharyn” is copied and saved to the DB as “1BOropharyn” NOTE the dates! March 14: Re-optimization for “1B Oropharyn”. New optimal fluences saved to DB. MLC motion control points for IMRT generated. Normal completion.

20. What happened? March 14 “Save all” is started. All new and modified data should be saved to the DB. In this process, data is sent to a holding area on the server, and not saved permanently until ALL data elements have been received. Data to be saved included: (1) fluence data, (2) DRRs and (3) MLC control points

21. What happened? March 14, 11 a.m. An error message is displayed. The user presses “Yes”, which begins a second, separate, save transaction. MLC control point data is moved to the holding area. The purpose of this error message is so that you can click `yes’ to proceed

22. What happened? March 14, 11.a.m. The DRR is, however, still locked into the faulty first attempt to save. This means the second save won’t be able to complete. The software would have appeared to be frozen. Ctrl-Alt-Del usually gets you out of this so....

24. What happened? March 14, 11.a.m. Within 12 s, another workstation is used to open the patients plan to load into VARIS and to treat. Unbeknownst to the users, the MLC motion file is NOT properly saved because the error message was over-ridden

25. What happened? March 14, 11 a.m. No verification plan, no pre-treatment dosimetry, no review by 2 nd physicist (not enough time!!) Several computer crashes ignored and over-ridden. Plan approved by physician (the plan looked OK on printout, but the computer data files were corrupted)

26. Big Errors are usually a team effort! So far: The radiation oncologist did not read port films on time, and then rushed everybody else Neither therapists nor physicists confront him The treatment planner over-rode an error message they did not understand The physicists did neither a double check, nor a dosimetry test of an IMRT plan (also called `billing fraud’) There were real bugs in the software design (it crashed, two people could open file at the same time) So far no real harm has been done! But they’re sowing the seeds for a really great accidents!

27. And now the therapists add the final piece! March 14, 2005, 1 p.m.

28. What they should have seen:

30. March 14, 2005, 1 p.m.: What they also didn’t notice: 1324 MU with MLC wide open

31. Discovery of accident March 14-16, 2005 The patient is treated without MLCs for 3 fractions On March 16, a verification plan is created and run on the treatment machine. The operator notices the absence of MLCs. A second verification plan is created and run with the same result. The patient plan is loaded and run, with the same result. Impact of accident The patient received 13 Gy per fraction for three fractions, i.e. 39 Gy in 3 fractions

32. Was there a bug in the Manufacuters software? Yes, but the software bug needed a lot of help in order to kill somebody: 1. MD didn’t check port films and rushed the plan modification 1. Nobody confronted the MD 2. Error messages ignored and not investigated 3. Treatment plan QA not performed 4. Therapists weren’t watching MLC display 5. neither the manufacturer, nor the user, nor the FDA, nor anyone else has the expertise to test this software for safety 6. Accident not made public until much later! Disasters are a team effort!

33. More radiation therapy accidents Springfield, Mo., 2004-09. 76 SRS patients overdosed >50 percent. Used too large a dosimeter to calibrate SRS fields. No independent check, no mandated state or federal reporting reguirement, no requirement for physicists or therapists to be certified. Insufficient support from manufacturer. Tampa, Fla., 2004-5., 77 SRS patients overdosed >50% because PDD factor not used in TG-51 calibration. Uncovered after 1 year during RPC inspection for participation in RTOG.

35. 10mm 24mm 7mm For 10mm cone The Farmer chamber is much larger than the x-ray field, and most of it is in the beam penumbra. Dose reading = (6+4x0.7)/24 = 37% Penumbra = 3-5 mm Penumbra = 3-5 mm Note: dose uniformity should be +3% over 80% of the field width (for 10 x 10 field size). 80% of field = 8mm. But for very small fields uniformity is worse.

36. More radiation therapy accidents Another radiosurgery error: Linac collimators not properly set for small SRS cones (hospital staff assumed this was done automatically) SRS cone Insert (2-3) SRS cone Holder (5x5) Linac Collimators (10x10)

37. More…... UK, 1982-90: incorrect SSD correction (did not know how TPS worked). 1045 patients, 30% underdose, >492 RT failures Bend, Oregon, 1980’s: incorrect TPC. 13% overdose Spain, 1990: Linac `repair’ led to 36MeV e- beam no matter what was programmed. No dosimetry check. 27 patients, 15 deaths France, 2004: incorrect MU for dynamic wedge. 23 patients overdosed 20%, 4 deaths Glasgow,2006: incorrect calculation of MU. Planner thought TPS calculated MU/Gy and not MU/fraction. It didn’t! 67% overdose results in death France, 2006-7: large ion chamber used for SRS. 145 overdoses.

38. Errors we have Seen at Memorial Sloan Kettering

39. MLC Error (FSRT) Event Sequence:  FSRS treatment scheduled right after DMLC treatment  At end of DMLC treatment leaves are all closed  BrainLab mMLC attached to Linac below regular MLC  Therapist does not retract primary MLC leaves  Primary MLC leaf position not detected by R&V (thinks it ’ s SRS)  Patient treated with correct mMLC apertures and closed primary MLC Why:  Software not designed for two MLC ’ s  Light field not normally used for SRS

40. DMLC Error: IMRT treatment with open MLC leaves:  DMLC field selected for treatment after a static MLC treatment  DMLC plan loaded, leaves retracted for light field use  “ Go ” selected, leaves fail to return to prescribed position  All systems allow treatment to proceed with retracted leaves  Therapist fails to detect error Why:  Software did anticipate this sequence of events  Could only have happened with exactly the right wrong timing  Very similar to Therac-25 disaster

41. Record/Verify Systematic Error with DMLC:  DMLC (v1) created from TPS and sent to R/V  Dosimetry checks done  Plan changed  MU (only) manually edited in R&V  No Independent Check of Data  Difference too subtle to see on Portal Image Check Why:  Human error  Improper understanding of software (change in TP system doesn ’ t automatically get transferred to R/V)

43. Upgraded,VARIS to ARIA

45. Radiation Therapy Safety Program at MSKCC MSK was the first center in the US to initiate IMRT Since 1995 we have treated over 20,000 patients with IMRT (> half a million treatment fractions) We never dismiss the potential for errors, and have put into place a multi-tiered Quality Assurance Program to continually monitor all phases of radiation therapy treatments at MSK Almost all radiation therapy accidents result from a combination of equipment (or software) malfunctions PLUS human error

46. Radiation Therapy Safety Program at MSKCC - training in all new technologies - staff all board certified and licensed - continuing staff education - all errors, large and small are fully discussed at monthly meetings of the Rad Onc QA Committee - Minutes of Rad Onc QA Committee are reported and discussed at the Hospitals QA Committee - QA program is continually modified and upgraded to reflect new findings and technologies - New equipment and computer systems are tested prior to release for patient treatments

47. The human component of most radiation therapy accidents results from a combination of: failure to appreciate limitations of new technologies inadequate redundency in QA program: - people checking people - people checking computers - computers checking people - computers checking computers static or non-proactive QA program

48. System wide QA of all treatment devices, hardware and software Patient specific QA for each individual patients customized treatment plan Quality Assurance at MSK consists of two equally important components :

49. System wide QA of all treatment devices, hardware and software Annual dosimetry intercalibration test with Radiological Physics Center Periodic dosimetry and treatment planning accreditation from RTOG for IMRT and IGRT national protocol studies Periodic reviews by ACR (ASTRO??)

50. Every day, during each treatment fraction for all patients all treatment parameters (dose, mlc settings, x-ray energy, etc. are recorded by the linac computer and stored in a `treatment log file’ Every evening a `batch’ computer program is run comparing all `IMRT treatment log files’ with the planned MLC files. Discrepancies are investigated immediately by a medical physicist On a weekly basis all patient treatment folders are reviewed by both a medical physicist and a radiation therapist to insure consistency between delivered treatments and treatment plans On a weekly basis portal films and/or orthogonal x-rays are taken on each patient to insure correctness of patient treatment position Using Linac Log Files

51. Human errors usually fall into one of three categories: 1. Department policy is properly followed, but an error occurs anyway. For example, policy requires that all treatment plans and MU calculations be independently checked by a second person before the patient's first treatment. This policy is followed, but the second person also fails to detect the error; 2. Department policy is not followed. For example, no one performs an independent check of treatment plan or dose calculation prior to the first treatment; and 3. Department policy is deficient or incomplete. Most common for new technology 4. Bizarre errors: sequence of events, almost impossible to foresee or prevent

52. Nothing is foolproof for the sufficiently talented fool! R/V systems, computer controlled Linacs, image guided patient positioning systems, etc. reduce but do no prevent errors. They enable humans to make different kinds of mistakes faster and more efficiently.

53. New Paradigm for QA Most errors are NOT systematic. They are patient specific. Therefore QA should shift from equipment focused to patient focused. Patient Specific QA :  Treatment plan check (more difficult than before)  R/V, file check-sums (each fraction)  Independent MU check, dosimetry, portal images  Log file analysis, chamber measurement, film dosimetry Machine Specific QA :  Film test  Dosimetry test  Drift test  MLC and IGRT tests

54. Is QA Reporting Like an Iceberg?

55. Reported Errors Discovered Errors Actual number of errors

56. The Aviation Approach Even good people make mistakes, they are the the rule, not the exception Mistakes result from flaws in the system Find the system flaw that made it possible for a particular individual to make a mistake Start at the top and work down the system until the cause of the error is found They rarely make the same Mistake twice The Hospital Approach We are Gods. Mistakes are the exceptions not the rule Mistakes result from individual screw ups Find and punish the person who made the mistake Start at the bottom and work up. Find the lowest ranking person you can blame

57. The Aviation Approach Even good people make mistakes, they are the the rule, not the exception Mistakes result from flaws in the system Find the system flaw that made it possible for a particular individual to make a mistake Start at the top and work down the system until the cause of the error is found They rarely make the same Mistake twice The Hospital Approach We are Gods. Mistakes are the exceptions not the rule Mistakes result from individual screw ups Find and punish the person who made the mistake Start at the bottom and work up. Find the lowest ranking person you can blame

58. The Aviation Approach Even good people make mistakes, they are the the rule, not the exception Mistakes result from flaws in the system Find the system flaw that made it possible for a particular individual to make a mistake Start at the top and work down the system until the cause of the error is found They rarely make the same Mistake twice The Hospital Approach We are Gods. Mistakes are the exceptions not the rule Mistakes result from individual screw ups Find and punish the person who made the mistake Start at the bottom and work up. Find the lowest ranking person you can blame

59. The Aviation Approach Even good people make mistakes, they are the the rule, not the exception Mistakes result from flaws in the system Find the system flaw that made it possible for a particular individual to make a mistake Start at the top and work down the system until the cause of the error is found They rarely make the same Mistake twice The Hospital Approach We are Gods. Mistakes are the exceptions not the rule Mistakes result from individual screw ups Find and punish the person who made the mistake Start at the bottom and work up. Find the lowest ranking person you can blame

60. The Aviation Approach Even good people make mistakes, they are the the rule, not the exception Mistakes result from flaws in the system Find the system flaw that made it possible for a particular individual to make a mistake Start at the top and work down the system until the cause of the error is found They rarely make the same Mistake twice The Hospital Approach We are Gods. Mistakes are the exceptions not the rule Mistakes result from individual screw ups Find and punish the person who made the mistake Start at the bottom and work up. Find the lowest ranking person you can blame

61. The Aviation Approach Even good people make mistakes, they are the the rule, not the exception Mistakes result from flaws in the system Find the system flaw that made it possible for a particular individual to make a mistake Start at the top and work down the system until the cause of the error is found They rarely make the same Mistake twice The Hospital Approach We are Gods. Mistakes are the exceptions not the rule Mistakes result from individual screw ups Find and punish the person who made the mistake Start at the bottom and work up. Find the lowest ranking person you can blame

62. The Aviation Approach Even good people make mistakes, they are the the rule, not the exception Mistakes result from flaws in the system Find the system flaw that made it possible for a particular individual to make a mistake Start at the top and work down the system until the cause of the error is found They rarely make the same Mistake twice The Hospital Approach We are Gods. Mistakes are the exceptions not the rule Mistakes result from individual screw ups Find and punish the person who made the mistake Start at the bottom and work up. Find the lowest ranking person you can blame

63. The Aviation Approach Even good people make mistakes, they are the the rule, not the exception Mistakes result from flaws in the system Find the system flaw that made it possible for a particular individual to make a mistake Start at the top and work down the system until the cause of the error is found They rarely make the same mistake twice The Hospital Approach We are Gods. Mistakes are the exceptions not the rule Mistakes result from individual screw ups Find and punish the person who made the mistake Start at the bottom and work up. Find the lowest ranking person you can blame

65. Use of High Tech In Surgery Position in the pecking order MD’s Nurses Technicians Number Of chances to misuse hi-tech e.g.; robotics, lasers, laproscopic

66. Use of Hi-Tech in RT: inverted training/culpability Pyramid MD’s Physicists Dosimetrists Therapists Therapist Dosimetrists Physicists MD’s e.g.; Linac, MLC, IGRT, R/V, treatment planning Position in the pecking order Number of chances to misuse hi-tech

67. Use of Hi-Tech in RT: inverted training/culpability Pyramid MD’s Physicists Dosimetrists Therapists Therapist Dosimetrists Physicists MD’s e.g.; Linac, MLC, IGRT, R/V, treatment planning Position in the pecking order Number of chances to misuse hi-tech