1. RANZCR Radiation Oncology Training Network and Training Program Curriculum Evaluation

2. Purpose of today’s session  Introduce the evaluation of the RANZCR radiation oncology training network and training program curriculum  Present survey findings (completed by DoTs, TNDs, ESOs and CSs prior to workshop)  Discuss survey findings –Small group work: General discussion about training networks including governance, sustainability, quality of the training experience. Identifying enablers and barriers to the effective implementation of training networks. –Large group discussion: Presentation of findings to the larger group.

3. The evaluation  The purpose of the project is to undertake an evaluation of RANZCR’s Radiation Oncology Training Networks and Training Program Curriculum  Terms of reference for the training network evaluation: Determine whether: 1.The network structure and governance is effective 2.The training networks meet the objectives as outlined in the DoHA contract (e.g. identify strategies around sustainability, advice on possible changes to governance arrangements etc) 3.The methods used to support the implementation of the training networks has been effective 4.Training networks have had a positive impact on addressing workforce issues 5.Training networks allow improved access to educational resources, including access to usage of web ‐ resources 6.The training networks are financially sustainable.  Terms of reference for the curriculum evaluation: Determine whether: 1.Trainees are satisfied with the curriculum and feel supported 2.Trainers, supervisors & assessors feel better supported under the new curriculum 3.The curriculum balances service delivery and education

4. The evaluation (continued)  Project commended in October 2012 and will be completed by February 2014  Stakeholder groups to be consulted as part of the evaluation include:  Trainees, Directors of Training (DoTs), Clinical Supervisors (CSs), Education Support Officers (ESOs), Training Network Directors (TNDs), Other clinicians (e.g. radiation oncology consultants), Medical Superintendents, RANZCR staff and the Chief Censor  Main data collection components of the evaluation include:  Document review (e.g. routinely collected data such as training site self assessment forms and network training activity reports, network progress reports to RANZCR)  Consultations (i.e. site visits, online surveys and telephone-based interviews)

5. The evaluation (continued)  Timing of consultations (July 2013 to November 2013)  Site visits will commence in the second half of 2013 and are likely to include:  Queensland: Princess Alexandra Hospital, Radiation Oncology Mater Centre, Royal Brisbane & Royal Women's Hospital, St Andrews Cancer Care Centre, Toowoomba Hospital  Victoria: Austin & Repatriation Cancer Centre, Heidelberg Hospital, Sunshine Hospital, William Buckland Radiotherapy Centre, The Alfred Hospital  Tasmania: Royal Hobart Hospital  SA: Adelaide Radiotherapy Centre, Royal Adelaide Hospital  NT: Alan Walker Cancer Centre, Northern Territory Radiation Oncology  NSW: Royal North Shore Hospital, Royal Prince Alfred Hospital, Westmead & Nepean Hospital, Liverpool & Campbelltown Hospitals South West Cancer Centre, Prince of Wales Hospital  WA: Telephone-based interviews only  Telephone-based interviews and online surveys will take place with sites which we are unable to visit or for stakeholders who are not available when site visits are taking place.

6. Pre-workshop survey about training networks  Governance  Sustainability  Workforce issues  Quality of the training experience  Accessibility to training resources  Support within training networks (for TNDs, DoTs, CSs and Trainees)  Trainee work readiness

7. Survey findings: Governance of training networks  Are NGCs effective? 19/26 respondents indicated networks are effective. Five comments about membership and two comments about NGC’s ability to effect change.  Have NGCs met their responsibilities? –overseeing training network operations (88% indicated ‘somewhat’ or ‘to a great extent’) –resolving local issues (72% indicated ‘somewhat’ or ‘to a great extent’) –developing the network training program (88% indicated ‘somewhat’ or ‘to a great extent’ –considering network sustainability issues (75% indicated ‘somewhat’ or ‘to a great extent’) –developing transparent processes for the management and oversight of training (85% indicated ‘somewhat’ or ‘to a great extent’) –taking part in the selection, recruitment and allocation of training terms and trainees across the network (78% indicated ‘somewhat’ or ‘to a great extent’)  How could governance arrangements be improved? Training network still new, some standardisation/formalisation of processes needed around the recruitment of trainees and induction, selection and succession planning for TNDs  Do NGCs regularly canvas views of network participants (trainees, DoTs, TNDs, CSs and ESOs)? 20/29 respondents indicated that NGCs do regularly canvas views, while 6 respondents indicated that views were not taken into account regularly enough.  Are training networks effective? 90% agreed there is a clear goal for training networks and 94% agreed there is investment in the training network of time, personnel and materials

8. Survey findings: Sustainability of training networks  Have TNDs and ESOs supported the set up and ongoing implementation of training networks? All respondents agreed that TNDs and ESOs have played an important role in the set- up and ongoing implementation of training networks.  What aspects of the TND and ESO roles are necessary for the ongoing implementation of training networks? Roles critical for ensuring curriculum requirements are met, coordinating sites and championing support for training networks. Leadership, communication, organisation skills and flexibility seen as important for roles.  What strategies could assist ongoing funding of these role? 6/14 respondents were unsure or not aware of strategies which could assist ongoing funding of the roles. 4 respondents indicated federal and/or local funding is needed to support roles.  What other activities could support the ongoing implementation of training networks? More training for TNDs and ESOs, more funding for trips to remote sites within networks and increasing number of ESOs.  Views on training networks? –79% agree that stakeholders (e.g. TNDs, DoTs, CSs, ESOs, trainees and other clinicians) are aware of the benefits of training networks –83% agree that training networks improved communication and relationships between sites –73% agree that the benefits of training networks outweigh costs

9. Survey findings: Workforce issues  What are the past and present workforce issues? Oversupply of registrars, reluctance of registrars to take up regional positions.  Have training networks addressed some of these issues? To some extent. Training networks seen as beneficial for addressing allocation of rotations to regional centres.

10. Survey findings: Quality of training experience Have training networks improved the quality of trainees learning experience? 54% indicated that training networks had improved the quality of trainees’ learning experience ‘to a great extent’, 39% ‘somewhat’ 36% indicated that training networks had fostered a culture in the workplace that is supportive of training ‘to a great extent’, 43% ‘somewhat’  Rotations within a network seen to be beneficial because they: –Expose trainees to different centres and ways of practicing –Allow trainees to work in different subspecialties (and learn different techniques) –Provide opportunities for trainees to experience different workplace cultures and work with patients with different cultural backgrounds

11. Survey findings: Accessibility to training resources  Have training networks improved access to educational resources and/or programs? –Tutorials –Seminars –Teaching courses –Mock examinations –Web resources  8/19 respondents reported that training networks helped formalise some training and facilitated access to and sharing of resources/programs across other sites within the network

12. Survey findings: Experience participating in a training network  How would you describe your experience participating in a training network? 23/28 had positive views; 1 had negative view  Do you feel better supported to deliver training as part of a training network? 18/27 respondents reported that they felt better supported to deliver training as part of a training network; 4 reported there was not change in the level of support received; 3 reported they did not feel better supported to deliver training as part of a training network.  What could be improved to better support TNDs, DoTs, CSs, ESOs and trainees? Secure ongoing funding, reduce size of networks, more protected time, improve communication between trainees, DoTs and networks, increase involvement of clinical supervisors in the training program.

13. Survey findings: Trainee work-readiness  Have training networks helped improved trainee work-readiness? –72% of respondents indicated that training networks have helped improve trainee work-readiness ‘somewhat’ and ‘to a great extent’. –18% of respondents felt that training networks did little or nothing at all to improve trainee work-readiness.  What aspects of training networks could be improved to support the development of work-ready radiation oncologists? Improve trainee selection, shift the focus from making trainees exam ready to work-ready (i.e. ensuring exams are a close representation of clinical practice), implement practice management courses for trainees.

14. Small group discussion Governance of training networks  How could the governance arrangements of training networks be improved? (how can they effect change locally? Would their terms of reference and/or membership need to change?)  What strategies could be put in place to ensure ongoing feedback by network participants is collected and discussed as part of ongoing quality improvement activities? Sustainability of training networks  What activities could be put in place to support the ongoing implementation of training networks? Quality of the training experience 93% of respondents indicated that training networks have improved the quality of trainees’ learning experience ‘somewhat” or ‘to a great extent’; 79% indicated that training networks have fostered a culture in the workplace which is supportive of training ‘somewhat” or ‘to a great extent  Can you describe the ways in which the quality of trainees’ learning experiences have improved? Can you describe how the workplace culture around training has improved? Enablers and barriers to training networks  Please refer to Handout: Enablers and barriers –Do you think this list of enablers and barriers is correct? –Are you aware of any other enablers and barriers exist?