1. Tutor experiences of training, support and teaching in Elluminate for the 2009B presentations of L120, L203 and L211 Dr Simon J Cross Learning and Teaching Development Team Institute of Educational Technology

2. Project Aims Associate Lecturer training prior to the course start – what training was useful? Planning and preparation for tutorials – how did ALs structure their tutorials and how were the tutorial dates arranged? Pedagogical design of tutorials – what were the challenges and solutions to designing effective tutorials using Elluminate and face-to-face tuition? Tutor experiences of delivering the tutorials – what was difficult to learn and how did students react? Associate Lecturer recommendations and suggestions – what advice and guidance could help future course teams, students and tutors in using Elluminate?

3. Training Provided (early 2009) Tutor Experience (2009B) Student Experience (2009B) Report Scope

4. Report Contents 62 Training programme and resources 7777 2.1 Tutor feedback 2.1.1 Usefulness of training materials 9 2.1.2 Issues with the training and support materials 11 2.1.3 Interview data 11 2.1.4 Suggestions for additional support and training 12 2.1.5 Time spent on the self-study materials 132.2 Tutor’s advice to new tutors 143 Teaching in Elluminate 143.1 Background literature 14 3.1.1 Synchronous audiographics at the OU 15 3.1.2 Broader perspectives 163.2 Structure of Tutorials 16 3.2.1 Arranging tutorials 17 3.2.2 Attendance at Elluminate and Face-to-face tutorials 19 3.2.3 Use of breakout rooms 203.3 Effectiveness of Elluminate for specific learning and teaching objectives 213.4 Issues with using Elluminate: differences between Elluminate and face-to-face tutorials 22 3.4.1 Loss of the visual 22 3.4.2 Technical issues 24 3.4.3 Emotional responses 24 3.4.4 Teaching time 25 3.4.5 Social Interaction 25 3.4.6 Changing teaching strategies 27 3.4.7 Additional tutor comments and advice 29 3.4.8 Using Elluminate outside the tutorial 303.5 Tutor reflections on teaching with Elluminate 30 3.5.1 Has Elluminate Enhanced the Quality of Teaching 30 3.5.2 How did students respond to Elluminate 31 3.5.3 Is Elluminate an effective way of conducting an ECA 31 3.5.4 Would tutors use Elluminate again if given a choice?

5. Respondent Profile Number of survey respondents L120: 18 L211: 7 L203: 12 Total: 37 Number Interviewed L120: 2 L203: 2

6. Respondent Profile Number of survey respondents L120: 18 L211: 7 L203: 12 Total: 37 Number Interviewed L120: 2 L203: 2 Profile 84% women 32% Over 60, 38% 51-60 years old 46% Significant Lyceum experience* 14% Some Lyceum experience 38% No prior experience 2 interviewed positive about software 2 interviewed neutral about software

7. Tutor Training I felt adequately supported in my planning and teaching with Elluminate: Agree/Somewhat Agree: 66% Disagree/Somewhat disagree: 22%

8. Tutor Training I felt adequately supported in my planning and teaching with Elluminate: Agree/Somewhat Agree: 66% Disagree/Somewhat disagree: 22% I had satisfactory amount of training in Elluminate before delivering the first tutorial: Agree/Somewhat Agree: 56% Disagree/Somewhat disagree: 36% I had satisfactory amount of training in Elluminate before delivering the first tutorial: Agree/Somewhat Agree: 56% Disagree/Somewhat disagree: 36%

9. Tutor Training I felt adequately supported in my planning and teaching with Elluminate: Agree/Somewhat Agree: 66% Disagree/Somewhat disagree: 22% I had satisfactory amount of training in Elluminate before delivering the first tutorial: Agree/Somewhat Agree: 56% Disagree/Somewhat disagree: 36% I had satisfactory amount of training in Elluminate before delivering the first tutorial: Agree/Somewhat Agree: 56% Disagree/Somewhat disagree: 36% Time spent on self-study materials Over 8 hours 51% 4-8 hours 13% 2-4 hours 31%

10. Tutor Training I felt adequately supported in my planning and teaching with Elluminate: Agree/Somewhat Agree: 66% Disagree/Somewhat disagree: 22% How useful were components Of the training and support [n=] Activity resources supplied by course teams: 19 (12) [37] Advice of course team or course briefing: 11 (17) [36] Previous F2F tutoring: 8 (15) [34] Other OU advice or staff advice on teaching in Elluminate or/and Elluminate champion: 11 (4) [22] Previous experience of using Lyceum: 10 (10) [23] External guidance (e.g. from Elluminate company): 5 (3) [13] I had satisfactory amount of training in Elluminate before delivering the first tutorial: Agree/Somewhat Agree: 56% Disagree/Somewhat disagree: 36% I had satisfactory amount of training in Elluminate before delivering the first tutorial: Agree/Somewhat Agree: 56% Disagree/Somewhat disagree: 36% Time spent on self-study materials Over 8 hours 51% 4-8 hours 13% 2-4 hours 31%

11. Tutor Training 'I gained most from Elluminate's self-study materials, I was able to do them in 'bite-sized' pieces at my own pace. The L120 team provided sessions which helped me consolidate this knowledge, and many individual problems were shared on the forum. The online ECA, which was my main worry, was well explained beforehand.’

12. Tutor Training 'I attended the on-line training sessions, which were well-conducted, but had already mastered much of the system. I always find it easier to learn by experimenting with the software. The on-line manual [probably the Elluminate manual] I did not find particularly user-friendly and there were features which just did not work at certain times near the start of the course.... another problem is, for somebody like me who has only one group, I have ample time to forget between classes.. I then feel terrible at having to keep asking the same stupid questions.’

13. Tutor Training: Suggestions and advice More hands on sessions Smaller online group training sessions Step-by-step guide focused on education and teaching & a reference sheet Individual support for those who just want to get on a learn it themselves Have more advice on how to Increase Elluminate attendance Ensure timing of activities not too late Better signpost materials Better ensure the supplied materials align with the assessment activities students will have to do

14. Tutor Training: Suggestions and advice Be prepared for problems (e.g. plan in time for resolving technical issues) and more rigorous, structured planning Individual support for those who just want to get on a learn it themselves Be prepared to spend some time using tutorial material and practising with it Learn basics first and then experiment: take things slowly and avoid technical overkill Don't hesitate in asking for support, keep calm and take all training available Encourage students to use it from the start and ensure they understand importance of testing before session starts Encourage students to use it from the start and ensure they understand importance of testing before session starts Encourage students to use it from the start and ensure they understand importance of testing before session starts Elluminate is different to Lyceum Anticipate student emotional responses: anxiety, stress, motivation, confidence…

15. Tutor Experience Strong sense that tutors had made great efforts to engage with the software Were tutorials effective in: Practice in speaking: 62% (19%) Practice in collaboration: 48% (32%) Etiquette of online collaboration: 43% (41%) Training in operating Elluminate: 38% (51%) Sense of being part of group: 38% (32%) Helping prepare for TMA/ECA: 38% (38%) Providing formative feedback: 35% (46%) Enhancing motivation: 27% (38%) Providing general support: 24% (38%) Resolving content problems: 14% (46%)

16. Tutor Experience Strong sense that tutors had made great efforts to engage with the software Elluminate was an effective way of conducting a speaking ECA Agree: 47% Disagree: 41% Were tutorials effective in: Practice in speaking: 62% (19%) Practice in collaboration: 48% (32%) Etiquette of online collaboration: 43% (41%) Training in operating Elluminate: 38% (51%) Sense of being part of group: 38% (32%) Helping prepare for TMA/ECA: 38% (38%) Providing formative feedback: 35% (46%) Enhancing motivation: 27% (38%) Providing general support: 24% (38%) Resolving content problems: 14% (46%) Problem with software: 97%

17. Tutor Experience Strong sense that tutors had made great efforts to engage with the software Elluminate was an effective way of conducting a speaking ECA Agree: 47% Disagree: 41% Would use Elluminate again in teaching if given the choice: Agree: 53% Disagree: 25% Neither Agree nor Disagree: 22% Were tutorials effective in: Practice in speaking: 62% (19%) Practice in collaboration: 48% (32%) Etiquette of online collaboration: 43% (41%) Training in operating Elluminate: 38% (51%) Sense of being part of group: 38% (32%) Helping prepare for TMA/ECA: 38% (38%) Providing formative feedback: 35% (46%) Enhancing motivation: 27% (38%) Providing general support: 24% (38%) Resolving content problems: 14% (46%) Problem with software: 97%

18. Student Experience End of Course Survey Satisfied with quality of face-to-face Tutorials I attended: 2009B: 84.5% - 89.5% - 87.0% Satisfied with quality of online tutorials/workshops took part in Using Lyceum or Elluminate: 2009B: 47.9% - 51.2% - 36.0%

19. Student Experience End of Course Survey Satisfied with quality of face-to-face Tutorials I attended: 2009B: 84.5% - 89.5% - 87.0% Satisfied with quality of online tutorials/workshops took part in Using Lyceum or Elluminate: 2009B: 47.9% - 51.2% - 36.0% Students responded well to Elluminate: Agree: 30% Disagree: 22% Neither agree nor disagree: 47%

20. Student Experience End of Course Survey Satisfied with quality of face-to-face Tutorials I attended: 2009B: 84.5% - 89.5% - 87.0% Satisfied with quality of online tutorials/workshops took part in Using Lyceum or Elluminate: 2009B: 47.9% - 51.2% - 36.0% Attendance at tutorials Students responded well to Elluminate: Agree: 30% Disagree: 22% Neither agree nor disagree: 47%

21. Student Experience End of Course Survey Satisfied with quality of face-to-face Tutorials I attended: 2009B: 84.5% - 89.5% - 87.0% Satisfied with quality of online tutorials/workshops took part in Using Lyceum or Elluminate: 2009B: 47.9% - 51.2% - 36.0% Attendance at tutorials 'Most' of the students attending F2F tutorials were also those attending Elluminate sessions: Agree/Somewhat: 70% (26 of 37) Disagree/Somewhat: 30% Students responded well to Elluminate: Agree: 30% Disagree: 22% Neither agree nor disagree: 47%

22. Elluminate compared to Face-to-face Subject of observation Number of responses on this subject Not being able to see the students both the individual student and the student group Loss of paralinguistic information: 20 Loss of ability to check comprehension: 7 Harder to interpret silences: 7 New pedagogic challenges: 6 Technical problems and difficultiesTotal: 22 Increased stress, anxiety and lack of confidence with technologyTotal: 9 Less can be taught in a comparable length of time because things take longer Total: 9 Less complex social interaction, cohesiveness and greater sense of isolation Total: 9 Lack of spontaneity and immediacyTotal: 5 Lack of familiarity with ElluminateTotal: 4 Great auxiliary support toolTotal: 2 The challenge of multi-taskingTotal: 2 Loss of ability to use physical resourcesTotal: 2 Ability to have breakout roomsTotal: 2 New opportunities to work with mediaTotal: 2 A tool for the ‘communicative people’Total: 2 MiscellaneousTotal: 10

23. Summary / Changing teaching strategies Greater use of breakout rooms New media opportunities but inclusivity issue Pragmatic and accept limitations Transferral or redesign? Role of F2F changing? Focus on task Next time will be better...