1. How best to evaluate the effectiveness of a VLE Chris Rowell Technology Enhanced Learning Advisor – Regent’s College, London.

2. Introduction Session outcomes:  Identify ways of evaluating technology enhanced learning  Understand how to evaluate a VLE using qualitative data  Identify ways in which effective academic practice in VLE’s can be supported and developed

3. The context Regent’s College: Central London 7 Schools Fees: £13, 000 to 21,000 4000 students 350 academics My role: learning technologist Bb VLE

4. How do you evaluate your VLE? Spend 10 mins discussing the following: Does your institution evaluate its VLE? How is your VLE evaluated? Who does it? Why do they do it?

5. Methodology Baume (2010) paraphrasing Kirkpatrick (1992):  Did people like it?  What have they learnt from it?  Have they applied what they learnt to their practice?  Have the results improved?

6. My research project at Regent’s College: Quantitative data: My research: 1.Online questionnaire 2.Interviews 3.Analysis/report

7. Questionnaire

8. Interview questions Your task: Arrange the interview question in order of importance.

9. Interviews Interview questions for lecturers Starting Bb 1.a) How easy was it for you to start adding content to your module area? b) What were the main difficulties you faced when creating a new module area in Bb (if any) 2. Have you experienced any technical difficulties using Bb? Staff/Student interaction 3. Do you think the students have an increased expectation of your availability since Bb was introduced? 4. Do you believe that Bb can replace a part of your face-to-face contact with your students? 5. Do you think that Bb affects the attendance in your lessons? 6. In your opinion, has Bb changed the quality of interaction with your students? 7. Has the introduction of Bb affected your relationship with students? Teaching 8. Do you think that Bb enables you to become more or less student-centred in your teaching? 9. Has Bb affected (or could affect) the way you teach in the classroom? 10. How effective is Bb at supporting collaborative activities on your course? Assessment 11. How has Bb affected the way you assess students work? 12. Have you changed the amount of feedback to students since the introduction of Bb? 13. Do you use Bb to evaluate how much work the students are doing on your module? 14. Do you use Bb to identify any students who might be “at risk” on your module? Workload 15. Has the introduction of Bb changed your overall workload? Future needs 16. Would you like more training or on-to-one support to develop your Bb module? If so, what areas of Bb would you like to develop? 17. Would you like to make any other comments about Bb?

10. Report

11. Revisiting our evaluation criteria  Did people like it?  What have they learnt from it?  Have they applied what they learnt to their practice?  Have the results improved?

12. Who gets the report?

13. Why bother?

14. Conclusion What I have got out of it:  Justification of what I am already doing  Influenced future direction of the VLE  Ticks SM’s boxes  Unintended consequences

15. Any questions?

16. Contact me: rowellc@regents.ac.uk