1. JISC LXP In-depth study of students’ experiences of e-learning Gráinne Conole, Maarten De Laat, Jonathan Darby and Teresa Dillon E-Learning Research Centre University of Southampton Email: g.c.conole@soton.ac.uk JISC Online conference, 27 th -31 st March 2006

2. Outline Rationale for the work Current Learning Experience projects LXP –Focus and aims –Methodology –Subject disciplines Methodological issues

3. Rationale for the work Sharpe et al (2005) –Major review of purported studies on students’ perspective –Found few studies actually really focused on the student –Main focus to date has been on practitioners and their use of e-learning –Little focus on students’ experience of e-learning –Specific methodological issues arise when researching student experience

4. Current JISC projects Two projects currently underway –LEX – Linda Creanor et al (Glasgow Caledonian University –LXP – Gráinne Conole et al (E-Learning Research Centre, Southampton University) Overarching scoping project –Review plus ongoing support – Rhona Sharpe et al (Oxford Brookes University)

5. Focus Primary aim is to distil out subject discipline issues in using e-learning Achieved by: –collecting data on students’ experiences of using e-learning To support specific learning activities Throughout a time period to support a range of learning-related activities –describing the students’ personal background and learning context –drawing out learner beliefs and e-learning strategies

6. Research questions How do learners engage with and experience e-learning? –What is their perception of e-learning? –What do e-learners do when they are learning with technology? –What strategies do e-learners use and what is effective? How does e-learning relate to and contribute to the whole learning experience? –How do learners manage to fit e-learning around their traditional learning activities?

7. Sample Working with four HE Academy subject centres: –Medicine, Dentistry and Veterinary Sciences –Economics –Information and Computer Sciences –Languages and linguistics

8. Data collection Broad brush survey across four subject disciplines –To gain a wider understanding of learner XP around particular artefacts and select effective e-learners for case studies Case studies of individual learners –Describing e-learning activities carried out by the learner –Exploring e-learner context and background Meta-study of case studies –Examine diversity of views in learning group Focus groups –Critical reflection on findings of the study of cases, cross checking with a wider group of e-learners and addressing gaps or emerging issues

9. First phase - survey Aim is to gain understanding of the setting in which learners learn and use technology to support their learning To gather general background information about the learners and the way they integrate technology into their learning –How different learners engage with e-learning and integrate technology into their learning –Learner background, selection of technologies, competence and ICT experience –Pedagogical approach, course design, and learning outcomes (artefacts) - from subject centres

10. Second phase – case studies Aim is to elicit students’ experiences with integrating technology in the learning 16-20 learners (4-5 learners per subject centre) for the in- depth case studies on their e-learning activities Mixture of observations and audio-logs to gather rich data around the artefact the learners are working on –Audio-logs diaries on their learning activities to provide rich data about the day-to-day events and a realistic account of the activities undertaken by the learners –Observations, including thinking-aloud techniques will provide evidence of the activities that students are engaged during e-learning Data collection starts in April, initial case studies include: –Computing Science – Ulster and Wolverhampton, focus on students mobile e-learning experiences –Medicine – Newcastle and Edinburgh, focus on VLE environments for practice based learning

11. Third phase – focus groups Aim is to allow for reflection upon e- learning experiences Used as a triangulation method to transcend from the data gathered from individual learners and validate and reflect on these results in a larger group of e- learners that were covered by the original sample For each subject area a focus group session will be held, consisting of 8-10 participants (including a number of participants of the case studies)

12. E-learning in the disciplines HE Academy/JISC symposium – 9 th February –http://www.heacademy.ac.uk/eLDisciplines.htmhttp://www.heacademy.ac.uk/eLDisciplines.htm –Aims were to reflect on how approaches to e-learning differ across the different subject and CETL communities identify how each community can best be supported to improve its e-learning provision

13. Social sciences Communication, dialogue –Face to face is integral and valued, even essential Practical engagement –with large body of evidence/resources – requires information literacy skills Employment related skills –Info management, historical skills, evaluating websites, time management Slides taken from http://www.heacademy.ac.uk/learningand teaching/SocialScience.ppthttp://www.heacademy.ac.uk/learning

14. Social sciences - Technology Not dialogue - online discussions difficult to do well Getting students using resources to explore background and context (large amount of material in law, topical in human rights, digital archive in history) Communications and groups support development of engaging with other cultures, working with international groups Tracking and monitoring of students supports students’ own management of learning skills (from time management to portfolio creation) Slides taken from http://www.heacademy.ac.uk/learningand teaching/SocialScience.ppthttp://www.heacademy.ac.uk/learning

15. Challenges Efficiency rationale for e-learning Access to technology for all students Facilitating online discussions Information literacy skills International groups Using web as an authoritative resource Need to respond very quickly Slides taken from http://www.heacademy.ac.uk/learningand teaching/SocialScience.ppthttp://www.heacademy.ac.uk/learning

16. Natural Sciences and Maths Curriculum issues –Students need to ‘do things long hand’ – eg equations or geo-spatial mapping –Problem solving skills important conceptualising / modelling problems developing / extrapolating solutions testing and reflecting on solution. –Simulations and ‘virtual fieldwork’ and the role of ‘real’ fieldwork - tactic (sensory) knowledge gained through field trips Slides taken from http://www.heacademy.ac.uk/learningand teaching/NaturaSciencesandMaths.dochttp://www.heacademy.ac.uk/learning

17. Natural Sciences and Maths Technology opportunities and challenges –Technology had opened up new subjects and approaches (eg the raise of GIS as a subject) but benefits have been matched by tensions about what is core subject knowledge –‘Visualising the abstract’ (from numeric sequences to geo-spatial data) was a opportunity created by technology –Natural Sciences using technology to provide access to ‘real’ data from remote areas –Varying use of e-assessment - large MCQ based but interest in more complex ways –Technologies which are not commonly used - e- Portfolios and reflective tools such as weblogs –Use of ‘lab books’ which could be online and reflective Slides taken from http://www.heacademy.ac.uk/learningand teaching/NaturaSciencesandMaths.dochttp://www.heacademy.ac.uk/learning

18. Health Sciences Outcomes –Teamwork and reflection important, as is inter- professional communication Challenges –Getting away from the expert model, i.e. empowering the learner to give their own viewpoint –Issue of sharing resources –Where do policies on use of ICT come from? Some hospital authorities try to lock down the use of new technologies so that students bringing in PDAs and data sticks may not be ‘usable’ with hospital systems Slides taken from http://www.heacademy.ac.uk/learningand teaching/ELDisciplinesHealthScience(1).dochttp://www.heacademy.ac.uk/learning

19. Health Sciences Challenges –Curriculum has to acknowledge competencies for practitioners that are set by governing bodies and professional bodies –People (staff) can get left behind in terms of skills expected of them. This has staff development implications. Also, staff without e- learning know-how will not be acting as a role model fro students Slides taken from http://www.heacademy.ac.uk/learningand teaching/ELDisciplinesHealthScience(1).dochttp://www.heacademy.ac.uk/learning

20. Audio diaries Three trial sample audio diaries are provided along with transcripts Will highlight the key findings which emerge from these in terms of students use of technology

21. Audio diary One Chinese female PhD student See technology as essential and integral to her studies Use of Internet to find information and for communication Online material to refer to and reflect Sharing of articles Technology integral across work and social life

22. Audio diary One Chinese female PhD student Difficulty of finding paper-based journals Subtle use of communication tools for different purposes – email less intrusive than the phone; mobile for urgent communication Video conferences for online seminars and motivational Lack of confidence with some research tools such as NVIVO Care in selecting appropriate resources; using time online effectively

23. Quotes “I think that technology is becoming more and more important for my studies” “I think is technology also provides a means for me to communicate with tutors and the other students and when we are doing this research training course in the university we have this blackboard and the tutors most of them not all of them some tutors put their notes on blackboard and we can always check back and we can download it before the course started so we can get ourselves familiar with the material and then when we come to the discussion of the course we feel it just make more sense the topic makes more sense to us”

24. Quotes “If I can get something online I will go for that first because it is always good you can get electronic form to keep in your computer and you can always trace back but for paper – and also you can print out if you want you can read it first and then if it is useful you can print out but if something is not so relevant you can just leave it” “[Communication] through email because I think everyone most other people check their email quite often and so that’s the yeh and also this is the sort of communication media is less interruptive for me because its not like you phone somebody and you are not sure if she or he or she is convenient or not at that moment but you email and he can always pick up when he got time”

25. Quotes “And you like through this media [Video Conferencing] you bring people together and because sometimes seeing people you can really boost kind of motivation thing because you know who you are talking to and you get more human interaction with it” “Yes sometimes I contact with the tutor [via mobile phone] cos like some tutors busy you have no other way to find them so mobile phone is the only way” “I remember this one day just something wrong with my computer with my internet connection the whole day because I couldn’t get internet so I think finally I sat down and I got a lot of work done so I think that might be one thing one of the caution you have to be careful”

26. Audio diary Two Saudi Arabian male PhD student Increased use of technology in recent years Easy to use and find information Value of online research articles Online international research seminars Flexibility of working approach Communicating with friends, tutors and scholars

27. Quotes “once I came here and I am doing my master and phd I started using the technology very very much and it was very helpful and easy and for collecting data and for receiving information” “I have attended more than four online conferences during the last three years and the last one it was an e-learning conference seminar … it was very helpful and the research of their website was the powerpoints which would be downloaded

28. Quotes “now I am using the internet *** connect my computer to the university and it work and so I do all my work online *** and this give me flexibility and sometimes I go back home to Saudi Arabia like for a holiday and I can manage my work from there and I can have access to my account here at the university and also it helps me to be connected with lots of friends and scholars and in the area of my subject through either the email messages or through the messengers system connect to my colleagues and so now I can use the technology play an important part in the different information or a way a good way in receiving information and in support learning nowadays”

29. Audio diary Three English male PhD student “My experience is that it [e-assessment] certainly helps with formative assessment so that one can test oneself against different parts of the curriculum. The downsides include lack of personal feedback so that you don’t necessarily know that what you are studying is what you should be studying. And also that there is only a limited amount of courseware available because the topic isn’t well provided for being so specialised. The upside of it is the ability to produce things very rapidly”

30. Key points All comfortable with using technology See it as integral for their learning Sophisticated and varied use of different communication tools for different purposes Access to up to date and relevant information and resources vital

31. Activities Further background documentation on the project can be found here.found here Complete the LXP survey and submit online (see the link in the conference). Try out some of the audio logs. Send in your own experience of e- learning, by posting a message in the conference. Use the associated discussion forum to consider the questions raised in this presentation.

32. Questions Scoping review –Sharpe et al’s review gives some suggestions as to why so much of the focus of research has been on the practitioner perspective – but raises the question: How can a better understanding of the learner perspective affect our current practice? LXP survey –Provide your comments on the survey –Do you think it is useful and what are its limitations? –What might we learn from the survey? –In what ways to you think findings from such a survey would be useful to practitioners?

33. Questions continued What specific methodological issues arise with researching students? Sharpe et al’s study recommended the ‘Interview plus’ method hence our use of audio logs around learning artefacts - what are the pros and cons of using audio logs? What other approaches could be used to gather and interpret data of this sort? What can we learn from audio files and transcripts? –What kinds of things might be interesting or surprising? –What might they tell us about designing for learning? –What they tell us about effectively supporting learners using technology?

34. Your experiences Disciplinary focus –How you think technologies might be used differently in your disciplines? –Do you think there is a mismatch between practitioner perceptions of the use of technology and the learner perspectives on this? –Can a better understanding of learner perspectives on using technology help us design better support for practitioners? –How will a knowledge of discipline differences help in terms of providing better support for learners, better learning designs and better tools? Share your experience of similar research work! –Research you have been involved with –Your experience of the use of technologies across different disciplines –Knowledge of related research work