Jeff Michalski, MD, MBA, FACR Professor and Vice-Chair Department of Radiation Oncology Washington University School of Medicine

Modern RT Recent sophistication – large fraction of modern treatment technology and practices developed in the past ten years High technical complexity Multiple systems (software and hardware) Limited to non-existent guidance/standards High pressure Increased potential for catastrophic failures

Current Processes Legacy processes and organization High reliance on manual checks and relatively poor use of electronic tools and automation Relatively Minimal Standardization Machine commissioning example – Reliance individual on facilities Almost Non-Existent Benchmarking Treatment planning example – Utilization of technology Poor quantification of processes

Error spectrum Publicized - One side of the spectrum, usually large dosimetric errors – NY Times Articles Semi-publicized – RPC data Approximately 30% of participating institutions fail to deliver IMRT dose indicated in their treatment plans to within 7% or 4mm to an anthropomorphic phantom (IJROBP. 2008;71(1 Suppl):S71-5). Unpublicized/unnoted – everyday occurrences “Small” dosimetric errors and geographic misses Suboptimal treatment plans (contouring and dose distributions) Care coordination issues Unnecessary treatment delays Jensen, Nellemann, and Overgaard, Tumor progression in waiting time for radiotherapy in head and neck cancer. Radiother Oncol, 2007, 84.

Errors in Radiation Oncology Staff and public exposures Suboptimal treatments Misadministrations Underdose Overdose Anatomical misses Magnitude From few percent to lethal doses From couple of millimeters to complete misses Regulatory Nuclear Regulatory Commission Errors that do not necessarily affect patients but have regulatory/legal consequences Sources Staff Software Hardware Random Affect one to few patients Systematic Affect hundreds of patients Potentially in a short period

Error Reporting in Radiation Oncology Use reported errors and near misses to develop safer software, hardware, tools, and processes

Background Global Problem “…it calls into question the integrity of hospital systems and their ability to pick up errors and the capability to make sustainable changes.” Sir Liam Donaldson, Chief Medical Officer, Department of Health Towards Safer Radiotherapy. London: The Royal College of Radiologists, Radiotherapy Risk Profile, Geneva: World Health Organization, 2009.

Systems Engineering in Health Care An outline for use of Systems Engineering for improvement of national health care system National Academy of Engineering and Institute of Medicine, 2005 “We often call this arrangement a “health care system” even though it was never created as a system and has never performed as a system.”

Error Reporting We are not airline industry nor nuclear power Perfection in complex systems across hundreds of diverse clinics is impossible Reporting systems for sake of reporting alone are a great way to squander resources and demoralize staff Error reporting as a part of broader process improvement efforts can be very valuable

Event Reporting Mandatory (statutory) Reporting required by law NRC in U.S. State requirements Mainly concentrated on well defined treatment delivery errors Guidelines for near-miss reporting typically not provided Voluntary Mainly at institutional level Some states in the U.S. have voluntary reporting systems – utility for rad onc not clear Errors and near misses tracked

Voluntary Reporting Dependent on Many Factors Culture Reporting guidelines Reporting system Competence to interpret reported data Willingness to implement, when necessary, significant changes based on collected data and subsequent analyses Ability to share the collected data and provide feedback

Reporting Culture Indemnity against disciplinarily proceedings and retribution Confidentiality To the extent practical, separation of those collecting the event data from those with the authority to impose disciplinary actions An efficient method for event submission A rapid, intelligent, and broadly available method for feedback to the reporting community

Taxonomy and Event Classification Event reporting should enable process improvement This requires efficient processing and analysis of data Submitted events must be classified and organized Enables efficient processing, analysis, and communication of data and trends

Web-Based Reporting

System Acceptance Paper

System Acceptance Voluntary Web-based Mutic et al, Submitted for publication, Med Phys, May 2010

System Acceptance Voluntary Web-based Mutic et al, Submitted for publication, Med Phys, May 2010

Therapy Nursing Physics Simulator Dosimetry

What are we doing? Organizational learning Error and near miss reporting Use reported events to improve processes and clinical tools Standardization QAIS QC – Based on reported errors and risk analysis Electronic Automated Intelligent Workflow tools - Based on reported errors and risk analysis

Electronic Chart Check Work Flow RT TPS Database OQA Database R&V Database Query Scripts dcm rtog ascii txt EcCk TPS & RV Prescription Field Parameters Consistency Reports etc Warning and Alerts Pending work Delivery problem indicators Independent checks Protocol Compliance Data Approvals, etc Statistics Benchmarking Future system improvements

Sample EcCk Report

Precheck scripts - workflow

Precheck: sample report

Plan Quality Benchmarking Moore et al, Submitted for publication, IJROBP, May 2010

Dynalog-based IMRT QA Report SummaryDelta FluenceMLC Graph Error Historical Performance of MLC

Tomotherapy: Sinogram based QA

The downside to the electronic world As implemented today Record and Verify (R&V) system was originally designed to operate as an independent system (Big Brother) Today these systems are integral part of the delivery process and the independent verification process is missing If data in the R&V system is wrong there is much less opportunity and chance that the error may be discovered

Possible Solution Electronic QA system (EQS) Independent system which compares TPS data with the data in the R&V system Greatly improves ability to compare initial data transfer and consistency of data in the R&V

Distributed Data Collection Each clinic with its own independent database Centralized Database ManufacturersRegulatory AgenciesProfessional Societies

Conclusion Error reporting in technologically advanced healthcare presents an opportunity to improve patient safety Independent oversight of established systems is desirable to monitor safety systems and implement immediate intervention Centralized reporting of errors and similar events (near misses, latent errors) is highly encouraged Use reported errors to improve commercial systems