1. Automated QA at Breast Cancer Rounds – A process to improve efficiency and quality of patient care Dr. Kathy Rock Breast Cancer Clinical Fellow Rock K, Barry A, McIntosh C, Purdie T, Koch C.A

2. Quality Assurance Quality assurance in radiation therapy is defined by the World Health Organization (WHO) as, “all procedures that ensure consistency of the medical prescription, and safe fulfillment of that prescription, as regards to the dose to the target volume, together with minimal dose to normal tissue, minimal exposure of personnel and adequate patient monitoring aimed at determining the end result of treatment” (WHO, 2008).

3. Consensus Statement 2011 Canadian Partnership for Quality Radiotherapy (CPQR) released statement in 2011 ‘all radiation treatment plans administered with adjuvant or curative intent…….. undergo radiation oncologist peer review of volumes and dosimetry ideally before the start of treatment in all cases, or if not possible, before 25% of the total prescribed dose has been delivered’’ Radiotherapy CPfQ. Quality assurance guidelines for Canadian Radiation Treatment Programs; 2013.

4. Background The peer review process in RT has been shown to detect errors that can be corrected prior to the delivery of the first treatment At Princess Margaret Cancer Centre on average 1000 radical breast plans and 300 boost plans are reviewed annually All radical breast plans are discussed at weekly Breast Cancer QA rounds (1 hour per week Thursday 12.30 – 1.30 pm)

5. 2223 Breast Cancer QA cases reviewed 2.1% underwent minor modifications 2.3% underwent major modifications

6. Objective Given large volume of cases the current challenge was to improve efficiency of QA rounds whilst maintaining quality and safety The objective was to develop an automated QA framework to better prioritize time for complex cases

7. Methods 1 st step All breast plans were assigned a clinical and planning complexity score during QA rounds Scores were assigned from July 2014 – February 2016

8. Methods Three groups 1. Breast/Chest Wall (CW) only 2. Breast/CW & Regional Lymph Nodes (RNI) 3. Boost A three point categorical scoring system was utilized – assigning scores for clinical and planning complexity

9. Breast/CW +/- RNI ScoreClinicalPlanning 0No discussion Standard case with standard fields/dose distribution 1 Minor discussion with RO consensus Minor change to standard fields – ie large separation, mixed beam energy, addition of anterior beam, reconstruction affecting dosimetry/pacemaker 2High complex/major discussion but overall consensus Complex planning/high dose to OARs/contralateral breast treatment/multi-field IMRT

10. Planning complexity 0

11. Planning complexity 1

12. Planning complexity 2

13. Boost plans ScoreClinicalPlanning 0No discussionStandard case 1 Minor discussion – over boost indication/seroma size Addition of beams/location of seroma/dose to OARs 2Major discussion – no seroma/oncoplastic surgery Complex planning - non co -planar beams

14. Boost – planning complexity 0

15. Boost - planning complexity 1

16. Boost – planning complexity 2

17. Methods 2 nd step Attributed scores reviewed by physics department Machine learning (SVM) was used to distinguish between the different complexity groups Automated ranking of clinical complexity was assessed using the normalized discounted cumulative gain. Assigned a perfect score of 1 if the predicted algorithm ranked patients exactly based on recorded clinical complexity

18. Results 375 plans reviewed using the normalized discounted cumulative gain Three possible orderings were considered 1. A completely random ordering 2. Breast/CW + RNI, Breast/CW only then boost plans 3. The algorithm’s predicted ordering

19. Results The metric assigns a perfect score of 1 if the predicted algorithm ranked patients exactly based on recorded clinical complexity Random 0.61 Breast/CW + RNI, Breast/CW only then boost plans 0.77 Learned algorithm 0.82

20. Conclusion Automated QA algorithm’s predicted ordering has led to superior ranking of complex clinical cases – learned algorithm 0.82 Demonstrated an ability of this automated QA process to rank and prioritize complex cases

21. Ultimately 3 rd Step will be integration of process prospectively into rounds and assessing its clinical applicability Increasing numbers  a process like this once fully developed could help to focus the QA rounds, simplify the process, reduce the potential for error and improve efficiency.

22. Acknowledgements 2016 CARO Resident or Fellow Scholarship The Phyllis Hantho Breast Cancer Fund Dr Anne Koch Dr Aisling Barry Dr Thomas Purdie Dr Christopher McIntosh Suzanne Lofgren Princess Margaret Cancer Centre and Radiation Medicine Program Breast Cancer Group

23. Automated QA at Breast Cancer Rounds – A process to improve efficiency and quality of patient care Dr. Kathy Rock Breast Cancer Clinical Fellow Rock K, Barry A, McIntosh C, Purdie T, Koch C.A