1. ALERT™ Education Outcomes and Impact on Clinical Practice Chris Parr Senior Information Analyst 1

2. What we hope this will do for you… Provide you with evidence of the impact the training has on patient care and justify your investment by demonstrating the value of the course. Enable you to quickly and easily report this evidence to your stakeholders (Patients, Trust Board, various departments of your trust, LETB, CCG). Contribute to your compliance to the Education Outcomes Framework. The HEE Mandate requires that there is a direct line of sight between education and training and the patient. 2

3. What we hope this will do for you… Allow you to identify any issues with the training whether they are local or course based. Allow you to compare the training before and after any changes are made. Instantly feedback to us so that we can make improvements/adjustments to the course. 3

4. The Surveys On The Day survey Post Course survey Both surveys were put together by a group made up of clinicians, L&D staff, and external survey experts. The questions are given weighted scores based on their importance and once collated give an overall score out of 100. 4

5. Results: On The Day Survey 5 Green Bars: ≥63 Programme exceeding standards Amber Bars: 46 - 62 Programme only just meeting standards, or falling slightly below standards Red Bars: ≤45 Programme falling below standards Data from 6 ALERT courses run between May 2014 to August 2014 100 25 86 / 23 / 22 / 19 /

6. Results: On The Day Survey 6 25 8 5 4

7. Results: On The Day Survey 7 25 4 5 8

8. Results: On The Day Survey 8 25 16 3

9. Results: On The Day Survey 9 25 5 10

11. 11

12. 12 Tick all that apply

13. 13 Tick all that apply

14. 14 Tick all that apply

15. Post Course survey 15 Key to measure the actual impact of the course. Designed to gain honest answers from the respondent focussing on what they do rather than what they know. Questions are predominantly frequency based (Never / Sometimes Usually / Always). Each score indicates the return on investment as the number of pence in the pound. For example, a category score of 89 indicates that the trust is getting a return on investment of 89 pence in the pound for that category

16. Results: Post Course survey 16 100 31 44 25 Overall Candidate

17. Results: Post Course survey 17 31 22 9 Sub-section 1 – Confidence and Capability Candidate

18. Results: Post Course survey 18 31 16 28 Sub-section 2 – Safe Practice Candidate

19. Results: Post Course survey 19 25 11 14 Sub-section 3 – Underpinning Knowledge Candidate

20. Collection Process On The Day survey collected on the day of the training course. Post Course survey collected via e-mail approximately 3 months after the training course. E-mail addresses of the trainee and their manager to be collected on the register and sent to PHT. Please promote the Post Course survey on the training day. 20

21. Collection process: On the Day Survey 21 +Everyone can complete a survey -Manual data entry required Paper Manual Data Entry QR Code Survey Website +Data feeds directly into system -Candidates need to have access to a smartphone / tablet -QR code reader app required -Possible difficulty typing correct web address Web Address www… :::::::::: :::::::::: -Accessible Wi-Fi or internet signal required Option 1 Option 2

22. 22 On The Day Evaluation Survey http://www.elesurvey.co.uk/f/612318/e847/

23. 23

24. Collection process: Post Course Survey 24 Candidate e-mail Candidate e-mail PHT system Candidate’s Manager e-mail Candidate’s Manager e-mail Registration to include… Link to Candidate survey sent to candidate e-mail www… Link to Manager survey sent to manager e-mail Link to Manager survey sent to manager e-mail www… 3 months later ?

25. 25 Downloading the results We will send you an EleSurvey link specific to your Trust.

26. 26 Downloading the results You can filter by date to report on a specific training day, a month, a quarter, or the year in full.

27. 27 Downloading the results Then export the scores to Excel.

28. Using the Excel Templates 28 We will send you 2 templates (1 for On the Day and 1 for Post Course) along with simple instructions on how to use them. Basically select all, copy and paste into the template. Then Save As. Then Save As PDF. Survey Website Excel Export PHT Excel Template PHT Excel Template PDF in printable format

29. 29 On the Day survey print preview

30. 30 Post Course survey print preview Candidate

31. Faculty / Peer Assessment This was one of the issues that we discovered and have subsequently resolved at PHT. This will be covered in Developing ALERT™ Faculty at 14:15. 31

32. Summary Identified benefits of using this evaluation process. Taken a look at example results from PHT. Mapped out the collection processes for both On The Day and Post Course. Talked about extracting the results. Shown what the print outs will look like. 32

33. Discussion Issues you foresee. e.g. Manager’s e-mail – best way to collect this? Possible solutions to these issues. Where would this data be best presented and to whom. How we can help make the process easier. 33