Presentation on theme: "A view from the auditor Melbourne 5th October 2012"— Presentation transcript:
1A view from the auditor Melbourne 5th October 2012 Patient Safety in Radiation Oncology: Australian EditionA view from the auditorMelbourne5th October 2012Ivan Williams, Joerg Lehmann, John Kenny,Jessica Lye, Leon Dunn
2Why use / trust an auditor ? So why would you let a bunch of guys (yokels?) like this into your facility to measure your equipment, and even if you did, why would you trust them?You, who have spend many days and weeks measuring, checking your very expensive equipment, with decades of experience and practice in the disciplines of medical physics, radiotherapy and radiation oncology, who are critically aware and concerned for the patient, what does an auditor, particularly the ACDS, offer?I submit to you that the answer has two parts, the first is a negative, defensive position based on risk avoidance, and fairly generic. The second is a positive approach, which is based on the improvement of the patients’ condition, to which I will provide explicit examples.
3Defensive (negative) rationale 1045 patientsThere are numerous examples of mis-treatments internationally. Many have been described in this workshop, and I won’t repeat the discussion. However, it is worth saying that the worst incidents are systemic, and have incorrect dosimetry as their root cause.Worst = longevity of mis-treatment and consequences of the error leading to 100’s of patients being mis-treated.
4International Context Numerous international examples of radiation oncology accidentsAnalysis has shown that most systemic errors could have been detected by independent external auditInternational audits have all found issues which needed to be resolvedKnow about Epinal, Staffordshire etc.SystemicBig errors usually due to basic dosimetry
5Why audit“Firstly and most directly, in every dosimetry audit programme, measured doses have been observed and reported which have been outside the required tolerances, in some cases significantly so.”Thwaites DI, SSDL 58, June 2010Izewska J, et al., SSDL 58, June 2011Ibbot, G.S., Followill, D.S., SSDL 58, June 2011.Know about Epinal, Staffordshire etc.SystemicBig errors usually due to basic dosimetry
6Oversight panels with professional experts Positive rationaleA properly constructed auditing program has resources beyond the capacity of any (many?) radiotherapy facilities.TimeDedicated staffDedicated equipmentTechnical expertiseOversight panels with professional expertsAudit is what these people do. The ACDS has a staff of around 4 efts, all dedicated to audit related issues all the time.The ACDS program has expertise and abilities beyond the capacity of many radiotherapy facilities within its area of audit
8Example II: Technical expertise Courtesy R.Ganesan, P Harty.
9Example III: Equipment ACDS recently purchased 7 new waterproof thimble chambers.ARPANSA calibrated them sequentially.All our audit kits have dedicated chambers, cables and electrometers.If we have a problem with one, it can be swapped out.Courtesy R.Ganesan, P Harty.
10Positive rationale: Oversight ACDSARPANSA (CEO)Branch HeadMedical RadiationDailyAdministrationFormalResponsibilityDepartment ofHealth and AgeingMOUCAGAuditorsFacilities & Professional OrganisationsAdviseApproval of DocumentsCAG comprises of experts drawn from all three professions.
11Technical Expertise: OSLD commissioning Al2O3:C (Sapphire) nanoDotTM OSLDsSize: 1x1x0.2 cm3Retractable Active element: 5 mm diameter, 0.2 mm thickMaterial: Aluminium Oxide doped with CarbonBar-coded for trackingSo, what are nanoDots? nanoDots are passive solid state dosimeters containing an amount of Aluminium oxide doped with carbon. The size of the overall dosimeter is 1 x 1 x 0.2 cm and inside the casing is a retractable sensitive element 5mm in diameter and 0.2 mm thick containing a layer of Al203 crystal sandwiched between to polyester films. The sensitive element is light sensitive and is only brought out of the case during readout. The outer casing is bar-coded allowing for tracking of individual dose histories.1.0 cm1.0 cm
12Commissioning - Overview ReproducibilitySignal depletion during readoutReader stabilityFadingLinearityEnergy dependenceSo now to the methods, I will only touch on the methodology briefly in this section however we do have a paper coming out shortly which details in full all of the methodologies associated with commissioning. So I’ll briefly touch on methods to determine the reproducibility, signal depletion per read, the reader stability, fading, linearity and the energy dependence.Why? Accurate calculation of absorbed doserequires a correction for each of these. Kf, Kl, Ke, Kr
13Signal depletion per read These are the results for the signal depletion per readout. A single nanoDot was repeatedly readout 190 times and the depletion in signal recorded. A linear fit to the data indicates a decrease of about 0.028% per read, which is consistent with findings in the literature of 0.03 – 0.04%. The lower depletion value obtained in our work is most likely due to a newer generation reader. The consequence of this is that each dosimeter can be re-read a number of times and depletion can be accounted for.Each readout of the dosimeterdepletes the signal by 0.028%.Consequence: Each dosimeter can be re-read a number oftimes and corrected to reduce readout uncertainty.
14LinearityDecrease in sensitivity as the absorbed dose to the nanoDot OSLD increases.The sensitivity = cGy per unit signal.Shown here is the linearity exhibited by nanoDots. We see that nanoDots exhibit a linear response up to approximately 3 Gy followed by a supralinear response at doses above this. The graph of sensitivity versus absorbed dose, where sensitivity is defined as cGy/signal is used to determine the linearity factor which is applied to the audit readings. In audit scenarios the accumulated dose to nanoDots does not exceed 3 Gy.Supralinear response with the degree of supralinearity being dependant on the accumulated dose.
15Energy Dependence 2.8 % w.r.t response at 6MV Slight energy dependence for MV photonsup to 1% relative to response at 6MVConsistent with (Aznar et al., 2004; Jursinic, 2007; Viamonte et al., 2008), along with the manufacturers stating that there is little to no energy dependence.Energy dependence is important in any dosimeter used for radiotherapy applications, we have determined the dependence for Co-60 6 , 10 and 18 MV photons, and a range of electron energies. There is a slight energy dependence for photon energies of about a percent with respect to the response at 6 MV. Low energy electron beams exhibit similar difference however a 2.8% discrepancy compared to 6MV was found for 20 MeV electrons. These results are consistent with reported findings and the manufacturers statement that there is little to no energy dependence.Electron beams show similar dependence(1 – 2%) with a larger measurement error associatedwith higher energies (18 – 20 MeV)
16Daudit = [(Counts.kr –Counts(bg).kr(bg).kf(bg)) kf ] ECF.S.kE.kL The absorbed dose can be directly calculated using the following equation:Daudit = [(Counts.kr –Counts(bg).kr(bg).kf(bg)) kf ] ECF.S.kE.kLECF is the individual element correction factor defined as the ratio of the mean batch counts, after 100 cGy irradiation, to the individual OSL counts, after 100 cGy irradiation.S is the batch sensitivity, in cGy/counts, to 100 cGy of 6 MV photons.kE is the energy correction to account for the slight energy dependence in OSLD response relative to 6 MV photons.kL is the non-linearity correction to account for the non-linear sensitivity of the OSLD, normalized to the sensitivity at 100 cGy.kr is the reader correction to account for the daily variation in the OSLD reader.A reader correction, kr(bg), is also applied to the background signal.kf is the fading correction to account for the reduced signal that occurs between irradiation and readout date.A fading correction, kf(bg), is also applied to the background signal from the initial ECF measurement. "So why are we determining all these factors, well aside from the obvious reason of characterising the dosimeter, in an audit analysis for Level I we take into account the element correction factor, the energy dependence, the linearity and fading as well as a reader correction for variation in the readers performance over time. Using this methodology, we have now initiated live photon beam audits of radiotherapy centres nationwide, with electrons on the way shortly, and if there is time ill briefly show you some preliminary data.
17Block FactorsThe dose delivered to the block, Daudit, is then converted to the equivalent dose under the reference conditions of the facility, Dref. The block dose requires corrections for; the different distance and depth between audit and reference conditions, the reduced scatter in the small block compared to a full scatter water phantom; and the difference between the block material and water.The BF was modelled with BEAMnrc/DOSXYZnrc and compared to measured block factors for the ACDS linac. The modelling of the BF was then extended to a range of beam qualities for each nominal photon and electron energy to cover the existing range of beam qualities in Australia.
19The ACDS: An auditing program In July 2010, the Australian Government funded a trial initiative to provide external, independent dosimetric verification for Australian radiotherapy centres: The Australian Clinical Dosimetry Service, ACDS. Housed within Australian Radiation Protection and Nuclear Safety Agency, ARPANSA, under a Memorandum of Understanding, MoU. Analysis of the service will be conducted in the third year to determine the outcomes of the ACDS A decision will be made whether to continue, modify or terminate the program based on the outcomesThere was debate about the model to be trialled, that selected was to utilise the dosimetric expertise at ARPANSA, rather than a clinical or academic environment.
20The service is free and voluntary ACDS trialFundamentallyTo increase the safety of radiotherapy within Australiavia:Three level audit – Level I, II and III.National coveragePrivate and public clinicsInteraction with professional colleges – Level IbThe service is free and voluntaryWithin MoUExtant to MoU
21Australian Context: Risk contributors Australians treated per yearOn-going roll-out of new radiotherapy clinics and updating of older machines – technologyDetection of errors within modern machines can be more difficult in modern cancer therapiesAustralian is BIG - logisticsSparse population and large cities – regional centres, country centres and some metropolitan centres do not have local supportStaff shortagesSo Australian radiotherapy, what’s going on?
22ACDS Audit Levels Diagnostic Imaging Target Outlining Treatment PlanningBeamCalibrationPatientSetupTreatmentDeliveryRecordand VerifyLevel IIILevel ILevel IILevel I: Linac output under reference conditions Level II: Treatment planning and delivery Level III: End-to-End testBased on T.Kron et al., IJROBP 52(2), 566–579, 2002
23Audit reporting based on auditors absolute measurement uncertainty () action level: 2, failed audit > 3reporting to center: “Dose is 0.7 % high with a 2 of 4.2%”2 sigma = = 95.4 %23
24Methodology & Design Dose to Patient Dose Delivery / Dose Calculation Level IAuditLevel IITreatmentDeliveryBeamModellingLevel IIILung /Thorax4DPlanning /IntensityModulatedH&NPelvisBreastPhantomgeometryCalcAlgorithmSimulationTargetingInhomogeneityDose to PatientDose Delivery /Dose CalculationLevel 2 includes multiple points of measurement in a beam enabling a Beam Quality parameter or in-beam match parameter to be determined.Dose to WaterAll audit rest on the fundamental dosimetry, however, as the investigation approaches the dose to patient, the multiple factors affecting the dose to the patient must be considered.
25Methodology & Design Dose to Patient Dose Delivery / Dose Calculation Lung /ThoraxDose to PatientH&NPelvisBreast4DPlanning /DeliveryIntensityModulatedLevel IIIAuditPhantomgeometryCalcAlgorithmSimulationTargetingInhomogeneityDose Delivery /Dose CalculationLevel IIAuditTreatmentDeliveryBeamModellingLevel 2 includes multiple points of measurement in a beam enabling a Beam Quality parameter or in-beam match parameter to be determined.Level IAuditDose to WaterWith an on-going audit program, such as the ACDS, a variety of Level II audit test capabilities provides a strong foundation for Level III audits and a fall-back approach when questionable Level III outcomes arise and must be investigated.
26Methodology & Design Dose to Patient Dose Delivery / Dose Calculation Lung /ThoraxDose to PatientH&NPelvisBreast4DPlanning /DeliveryIntensityModulatedLevel IIIAuditPhantomgeometryCalcAlgorithmSimulationTargetingInhomogeneityDose Delivery /Dose CalculationLevel IIAuditTreatmentDeliveryBeamModellingLevel 2 includes multiple points of measurement in a beam enabling a Beam Quality parameter or in-beam match parameter to be determined.Level IAuditDose to WaterSimilarly, issues arising with a Level II audit may be investigated and resolved with a Level I
27Level IPassive dosimeter, TLD/OSLD, placed in the clinical beam in a regular, reproducible environment with well understood conditions. External audit.Was TLD (IAEA approach), changing to OSLD for logistical and operational reasons in July 2012Required: 60% of all linacs in Australia.To-date: ~50%Expected: ~100%
28Not shown: beam quality measurements, where we found action level 6 MVMV18 MV
30Level Ib – by consumer demand On-site measurement with chamber for photons and electronsRequired in many European Nations,Required by Australian Radiation Oncology Practice Standards, criterion 15.1.Recombination, polarity and outputOrganisation supplies water tank, beam dataACDS supplies chambers, electrometer, meters, cables ...
32Original audits only, Not showing results of repeats, one repeat had OT now action level (electrons) and additional action level (photon)6 MV≥ 10 MV6 MeV8-9 MeV12 MeV15-16 MeV≥ 18 MeV
33Initial uncertainty did not include correlated uncertainty and used common factors for photon and electrons6 MV≥ 10 MV6 MeV8-9 MeV12 MeV15-16 MeV≥ 18 MeV
34Level IIDiagnosticImaging3D TreatmentPlanningPatientSetupTreatmentDeliveryRecordand Verify2D array of detectors placed in the clinical beam in a phantom of solid water. Lung slabs are added and measurement is compared with predictions from the computer planning system. Outcomes are derived from the spatial and dosimetric difference between the predicted and measured doses.The planning computer is given synthetic CT data which is used in the planning process. This ensures that issues arising from the CT process will not confound the outcome.
35Level II – Basic Design Required: 40 % of all linacs in Australia. Explain phantom, show lungs and point out measurement locations “This is what it looks like”Required: 40 % of all linacs in Australia.To-date: Testing and field trialsExpected: ~40 %
36Level IIIEntire process check from CT to treatment with a human-like plastic phantom. Outcome is obtained from the spatial and dosimetric difference between measurement and prediction.Required: 15 linacs within Australia.To-date: 9 linacs auditedExpected: 20+ linacsThe planning computer is given synthetic CT data which is used in the planning process. This ensures that issues arising from the CT process will not confound the outcome.
37Level IIIHumanoid Phantom (Ann D Roger) goes through the complete chain of procedures a patient experiences in Radiation Therapy.DiagnosticImaging3D TreatmentPlanningPatientSetupTreatmentDeliveryRecordand Verify110Explain phantom, show lungs and point out measurement locations “This is what it looks like”CIRS thorax phantom37
38Level IIIRadiation Therapists should conduct each of the steps in keeping with routine clinical practice so that the audit assesses the actual patient process.We want the people who do this day by day, not physicist, not the charge RT
39Level III Dose Tolerances measurement uncertainty () cannot be determined with sufficient accuracy in the given complex geometryclinical acceptability (5%) is used as a starting point for 3points in low dose areas / clinically insignificant areas / not well defined areas are reported but not scored (RNS) and reporting will be re-evaluated over time as data comes in with the goal of catching outlying results2 sigma = = 95.4 %39
40Level III – case 2 adjusted for points in low dose areas LocationExpected dosePlan vs MX (local ref)Plan vs MX (global ref)Point 1 – WDT~200 cGy-0.63% (-2.51%, +1.22%)Point 4 – WDT> 300 cGy0.06% (-2.62%, +1.65%)0.11% (-4.02%, +2.56%)Point 7 – LAA~4 cGy-17.1% (-42.9%, +18.3%)-0.37% (-1.02%, +0.34%)average (min, max)110Field: 6x, 10 cm x 15 cm 45° wedge Prescription: 2 Gy to Point 1 adjusted for points in low dose areasGlobal reference is used instead of local
41Level II - Field Description Level II underpinning Level IIILevel IIField IDLevel II - Field DescriptionLevel III referenceField 7case 3 LATopen field,asymmetricField 8wedged field,Field 9inhomogenityField 10wedged fieldInhomogenityExample fieldsACDS Reference Conditions. Measurement depth condition adjusted by adding orremoving slabs of solid water from upper layer.
421 % error in pressure ~ 1 % error in dose Recommendation IReview the accuracy of all barometers used for clinical dosimetry: Ensure that they are calibrated by a NATA accredited service, which is accredited for barometers.Ensure that the barometer(s?) is re-calibrated according to instructions.1 % error in pressure ~ 1 % error in dose1% pressure inaccuracy = 1 % dosimetric inaccuracy.Natjonal association of testing authorities.
43Early lessons learned Air pressure issues Barometer not calibrated (properly) or faultyAirport pressureEquipment (ionization chambers) problemsOutdated stylesSlightly damaged Solvable administrative problemsUnderstanding of calibration / QA processStaff changesLong living spread sheets – routine QAWrong calibration factor used
44ACDS Success FactorsMoU defined audit targets – have been shown to be flexibleInvitations to conferences, informal vocal support personally and at higher levelPositive outcomes from review – recommendation to continue in existing or near existing formatStart planning now – written document for reviewsMoU defines these, having said that, interpretation of the MoU is allowable and on-going.
45Successes & Challenges Level I – ACDS will overshoot 100*% v 60% (accepted by DoHA)Level Ib – Outside MoU (accepted (commended?) by DoHA)Level II – ensure the ACDS hits target of 40 %IMRT – Require plan for futurePrepare for review = prepare for post 2014External Professional Expectations/DesiresLevel I overshoot (100*% v 60%) feeds into the long term plan, discussed within CAG, that the three audits will eventually be delivered over a three cycle, one per year.Ib has huge support from the professions, discussion in RORIC hinted that PPP sites would have an ACDS type Ib audit as a prequisite within the provider contract.Formalise annual ARPANSA meetings with AIR, RANZCR and ACPSEM – annual conference?100 % = >95 %
46AcknowledgementsJohn Kenny Jörg Lehmann Leon Dunn Jessica Lye Tomas Kron Abel MacDonald Alison McWhirter Tracey RumbleRamanathann GanesanPeter HartyDavid WebbDuncan ButlerChris OliverPeter JohnstonThere was debate about the model to be trialled, that selected was to utilise the dosimetric expertise at ARPANSA, rather than a clinical or academic environment.'The Australian Clinical Dosimetry Service is a joint initiative between the Department of Health and Ageing and the Australian Radiation Protection andNuclear Safety Agency'46