1. Improving lecture capture through usability testing and research Ilkka Kukkonen @ilkkakukkonen #mlconf13 Media & Learning 2013, Brussels 12.11.2013

2. Motivation Create the best possible learning environment for our students and clients Improve overall quality, commitment and effects with optimal resource allocation Support personal learning paths and flexibility in participation Improve the pedagogical models and spesifically the lecture format Create multiaccess courses

3. INPUT & OUTPUT

4. Project approach to support the development (”MOVIE”, 2011-2012) N~800 The development of pedagogical quality and effectiveness of lecture capture –How the students are using the recorded lectures to support their studies? –How can we measure of the impact in various contexts? –How the pedagogical quality of the lecture capture can be improved? Data from teacher training and high school students, collected in mathematical subjects and statistics

5. A B C D 57% 49% 50% 64% A. Possibility for independent studies B. Retention and repetition of classes C. Illuminated unclear themes from lectures D. Helped to prepare for final exams Encouraging results (N=113) * 81% would like to increase the use of lecture capture in their studies 85% would recommend lecture capture for their peers * www.aducate.fi/movie

6. Findings from the study Students seem willing to increase the use of lecture capture in their studies Use of LC is seen as democratic method for participation It takes time to develop personal study strategies and build trust for LC Impact on lecture attendance seems to depend on the personal study strategies and previous experiences Viewing a lecture capture can feel more personal pedagogical experience than attending live lecture * www.aducate.fi/movie

7. Findings from the study Lecture capture seems to be more efficient for lower achieving students –Supporting conceptual understanding for content, metacognition and efficient study strategies Viewing volumes seem to have correlation to experienced value and grades Challenge is to create meaningful learning contexts for students –Include lecture capture in learning design –Summarize, contextualize and engage interaction * www.aducate.fi/movie

8. (CC) Liz Castro @ Flickr

10. INFORMATION DELIVERY VS. TEACHING

13. Usability study (3-6/2012, N=16)

14. Need to create realistic insight about LC use in genuine context Are there some problems related to the usability and content? How does the interface and features support viewing? How students find the essential parts of the lecture? 4 short lectures with different statistical assignments Familiarity of the content Pair work vs. individual performance

15. ● Research setup records students’ eye movement, video and audio and captures also mouse clicks (Tobii Eye Tracker) ● Two similar and syncronized workstations for pair work ● Downside is research setup is quite complex and annotation and analyzing the final data is time consuming ● Minor inaccuracies may occur within the people with glasses ● Supplemental interviews and survey Eye-tracking for lecture capture

16. VIDEO EXAMPLE

17. SLIDES LECTURER NAVIGATION TIMELINE

19. TEACHERS COMMENTS PERSONAL BOOKMARKS

20. Navigation Timeline Lecturer % of total viewing time Slides Total viewing time: regions

21. % of users used at least once Used features Scenes Timeline Forward / Reverse Personal bookmarks Teacher’s comments

23. Time in minutes Standard deviation Mean Time with assignments Recording 1Recording 2Recording 3Recording 4* ”Flip”

24. Question 3 Question 1 Question 2 Recording 1Recording 2Recording 3 Recording 4 % answers correct Performance * ”Flip”

25. Results from the video annotation (N=16) Teacher’s comments and remarks seem to support the viewing experience (scaffolding) Familiarity of the content and student’s personal notes and bookmarks speeded up completing the assignments Writing lecture notes while playing the video in the background gave lower scores in individual level Learning by copying the instructor, ”vicarious learning” (Bandura) seems to lead into surface level learning, which generated lower scores individual level and pair work Students who spent time with making sense of the subject got better grades than expected

26. ● Students formed pairs independently based on their social relationships ● Pairs typically shared the tasks for viewing and making notes ● ”Weaker” party asks and questions the answers, so the talented one needs to answer with thought and provide a convincing explanation about the potential answer ● In this setting stronger one seems to develop the weaker student’s grades ● Also in the final exam the weaker students outperformed the expected grades, so working in pairs with lecture capture might have developed their skills and motivation Working in pairs with lecture capture

27. www.aducate.fi www.uef.fi Thanks! Talk more? Collaborate? http://aducate.fi/movie ilkka.kukkonen@uef.fi @ilkkakukkonen