1. Food and Drug Administration Public Meeting June 9-10: Quality Assurance of Therapeutic Medical Devices In Radiation Oncology Nabil Adnani, Ph.D., DABR D3 Radiation Services Pittsburgh, PA.

2. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com Credentials Certified by the American Board of Radiology. Senior Medical Physicist and Chief Products Development Officer, D3 Radiation Services (University of Pittsburgh Medical Center). Active in the clinical and product development aspects of radiation oncology. Unique perspective as both developer (designer) and clinical user. Main focus is on minimizing errors in radiation dose delivery through well crafted processes and tools designed to implement them.

3. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com Questions seeking answers Q1: Is there a model QA program that exists which is widely accepted? If so, please describe. Q2: What types of QA should be the responsibility of the facility, the physicist, the operator, others? Q3: Should manufacturers provide QA procedures to medical facilities and users of radiation therapy devices? If so, why, and what instructions should be provided? If not, why not? How extensive should they be? Q4: Should Manufacturers provide training on QA practices? If so, why, what type of training should be provided, and to which personnel? If no, why not and who should?

4. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com No model of QA program exists in radiation oncology. Several organizations active in the field, such as AAPM & ACR, provide specific QA recommendations for specific treatment delivery technologies. Quality Assurance in radiation oncology is not a set of procedures to be implemented but a process designed to verify, validate and document that radiation dose is delivered safely and as accurately as possible. The process begins with the decision to implement a treatment delivery technology. The process should cover all aspects of the treatment planning and delivery: Device commissioning, TPS beam modeling, patient selection, patient simulation (or patient computer model), treatment plan, patient setup, dose delivery, etc. The process should document the nature of the verification, the validation methods and the results obtained. The process should be reviewed by qualified independent parties before going clinical. Q1: Is there a model QA program that exists which is widely accepted? If so, please describeIs there a model QA program that exists which is widely accepted? If so, please describe

5. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com The process should be able to keep pace and adapt to new developments in the treatment delivery technologies. The process should clearly distinguish what constitutes a QA and what does not. For example: What constitutes a proper chart check when the facility relies only on the R&V system to record actually delivered vs prescribed treatments? What are the measurements, verification and validation steps to follow in order to fully commission a linear accelerator for clinical dose delivery? How to standardize the documentation of all the methods used and the results obtained? Make clear distinctions between verifying that a dose will be delivered as intended vs. validating that it is actually delivered as intended: Calculations vs. Measurements. Q1: Is there a model QA program that exists which is widely accepted? If so, please describe

6. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com The safe and accurate delivery of prescribed radiation dose should be the responsibility of the qualified medical physicist. Given their training and knowledge, qualified medical physicists should establish the Quality Assurance Process and oversee its implementation. The facility and regulators are responsible for providing the environment in which qualified medical physicists can fulfill this critical component of their daily responsibility. There is some confusion in current clinical practices regarding the responsibility of the physicist with respect to computer systems used to control therapy devices. The R&V systems being one example. These are class II therapeutic devices. IT departments should assist physicists and not take over the management of these devices. Q2: What types of QA should be the responsibility of the facility, the physicist, the operator, others?

7. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com Manufacturers are in the best position to describe the procedures that can be used to ensure that their equipment meets established quality assurance standards or a standard/specification required by the purchasing clinic. Manufacturers have the responsibility to train the treatment team in the safe and proper use of their devices. What constitutes proper training should be determined by the clinic as part of their purchase agreement. Manufacturer should rely on training and support staff with adequate level of training, education and clinical experience. Manufacturer supplied guidelines should only be used as reference but not as a replacement to a well designed QA program by the facility’s qualified medical physicist. Q3: Should manufacturers provide QA procedures to medical facilities and users of radiation therapy devices?

8. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com Q4: Should Manufacturers provide training on QA practices? The role of the manufacturers should be limited to providing service, support and training on the proper and safe use of their systems. Qualified Medical Physicists should be responsible for setting up QA processes as well as training the rest of the staff on how to implement them. Manufacturers may/should seek the advice of radiation oncology professionals on what constitutes an optimal design capable of meeting the requirements of any QA process (already in place).

9. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com According to national and international organizations (IAEA, WHO, RPC from MD Anderson), the majority of reported events in radiation oncology can be traced back to errors committed during linac commissioning. We need to focus on providing tools and solutions for radiation oncology clinics to minimize errors at every stage with special emphasis on commissioning 1. The provision of such tools is not sufficient. It should be complemented by some form of a routine external audit of the entire QA Processes in place: Commissioning Audits, Beam Data Audits 2, QA procedures, treatment planning, delivery processes, etc.Beam Data Audits Most importantly, before going clinical, the QA process should pass the test of a third party review or reviews (if different levels of expertise are needed). 1 N. Adnani, “Design and clinical implementation of a TG-106 compliant linear accelerator data management system and MU Calculator,” Journal of Applied Clinical Medical Physics (In Press). 2 Online Beam Data Audit System or eDataAudit at http://www.d3cdms.comhttp://www.d3cdms.com Quality Assurance in Radiation Oncology: what we are missing…

10. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com The lack of reimbursement incentives means that the purchase of tools designed to ensure and maintain quality is often denied. Quality Assurance in Radiation Oncology: the weak link

11. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com Thank you

12. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com Common Commissioning Errors: I. Measurements Not understanding the requirements of the planning system’s algorithm. Scanning detector not appropriate for the size of the radiation field. Field and reference chambers are not of the same volume. Not shifting or Double shifting PDDs upstream per TG-51. Scanning along the chamber axis. Wrong definition of measured parameter. Wrong electrometer reading during OF measurements. Measuring the right OF but entering a wrong value (typo).

13. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com Common Commissioning Errors: II. Beam Modeling TPS beam model and the treatment machine calibrations are different Miscalibrating the beam model. Over-processing the data. Not verifying (auditing) processed data prior to using it for modeling. Importing the wrong data to the wrong energy or accessory. Errors in Dose Rate Tables inherited from measurement errors. Smallest field size measured not small enough. Wrong MLC transmission. Wrong dosimetric leaf gap (Varian Linacs).

14. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com IAEA Report: http://www-pub.iaea.org/MTCD/Publications/PDF/Pub1084_web.pdf Published in the year 2000 A collection of misadministration events

15. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com IAEA Report: http://www-pub.iaea.org/MTCD/Publications/PDF/Pub1084_web.pdf

16. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com IAEA Report: DescriptionInitiating EventContributing Factors Inconsistent sets of basic machine data Incorrect data for patient dose calculations Lack of or ineffective procedures, protocols and documentation Incorrect data for tissue maximum ratios Incorrect data for patient dose calculations. Insufficient safety provisions: Tables of measured tissue maximum ratios were not verified before clinical use. Insufficient understanding of the planning system algorithm Incorrect data for patient dose calculations Lack of or ineffective procedures, protocols and documentation: Commissioning with incomplete validation of the computer TPS. Incorrect basic data in a TPSIncorrect data for patient dose calculations (incorrect basic data used in a TPS). Lack of or ineffective procedures, protocols and documentation: Inadequate commissioning (TPS) Incorrect depth dose dataIncorrect data for patient dose calculations: The manufacturer provided basic data that were incorrect for some field sizes and depths. Lack of or ineffective procedures, protocols and documentation: Incorrect commissioning (data generated by the manufacturer were accepted for treatment, although the institution's physicist found an 8% discrepancy when he compared these data with his measurements). Error in the wedge factorAn incorrect procedure was used for computerized treatment planning involving wedges. Lack of or ineffective procedures, accepted protocols and documentation. http://www-pub.iaea.org/MTCD/Publications/PDF/Pub1084_web.pdf

17. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com WHO Report: http://www-pub.iaea.org/MTCD/Publications/PDF/Pub1084_web.pdf Published in the year 2008 Based on a literature review of reported incidents between 1976 and 2007

18. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com World Health Organization Report: Incidents from 1976 to 2007 http://www.who.int/patientsafety/activities/technical/radiotherapy_risk_profile.pdf

19. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com World Health Organization Report: Incidents from 1976 to 2007

20. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com Recent RPC Study Findings * : Pass Criteria: 7% or 4 mm DTA *Int. J. Radiation Oncology Biol. Phys., Vol. 71, No. 1, Supplement, pp. S71–S75, 2008 H&NProstateThoraxLiver Phantom Irradiations25473306 Pass17955173 Fail71971 % Failed28%12.3%23.3%16.6%

21. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com Incorrect output factors. Incorrect percentage depth dose. Inadequate modeling of the penumbra at multileaf collimator leaf ends. Incorrect application of QA calculations or measurements. Inadequate QA of multileaf collimator. Incorrect patient positioning. Errors in treatment-planning software. *Int. J. Radiation Oncology Biol. Phys., Vol. 71, No. 1, Supplement, pp. S71–S75, 2008 Recent RPC Study Findings*: Identified Problems Commissioning related Back

22. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com Linac Commissioning Process Components In an attempt to minimize errors in beam data and comply with AAPM TG-106 recommendations, D3 Radiation Planning implemented a Comprehensive Linac Physics Data Management System (CDMS) comprising the following components: Data management: Comply with TG-106 and minimize errors due to manual handling and processing. Data acquisition: Simplify data acquisition, recording and minimize errors by providing on-time comparison to reference/expected data. Acquired measurements become baseline data for TG-142. Data Auditing: Perform audits (TG-106), beam matching validation, peer reviews (TG- 103, Maintenance of Certification), etc. Data Book Generator: Comply with TG-106. Also, eliminates the “cut & paste errors” syndrome. Report Generator: Comply with TG-106, TG-142, TG-103 and state & federal regulations. Data Sharing or Communication Tools: Allows entire treatment machine data sets and associated documentation to be shared among system’s users. Useful for Peer Review (TG-103) and, critical, when a commissioning job is performed by a team of physicists. Linac Calibration: Comply with TG-142 and helps implement TG-51. MU Calculations: Serves as a truly independent MU Calculator. It is ready to use immediately following data acquisition. It is also useful during beam modeling for calibration and model verification (per TG-53).

23. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com Commissioning Process

24. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com Implement Commissioning Process

25. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com Clinical Study A total of 22 commissioning projects were analyzed in terms of the commissioning quality (error minimization and process flow) ranging from data collection and beam modeling to the clinic's feedback and satisfaction level. Out of the 22, 12 were completed without, and 10 with, the use of CDMS. As expected, errors in data collection were drastically reduced. Errors in data reporting (data book or hand calculation book) were also significantly minimized. Beam modeling errors have been all but eliminated. The overall satisfaction level with the commissioning work improved by a factor of 2 (200%).

26. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com Results Back

27. 5750 Centre Ave. Suite 500 Pittsburgh, PA 15206 http://www.d3cdms.com Sun Tzu: The Art of War Back “Strategy without tactics is the long road to victory; tactics without strategy is the noise before defeat.”