1. Presented by Marie Lippens
Engaging faculty in learning analytics. There’s a qualitative side too!
Presented by Marie Lippens

2. Outline What is Learning Analytics Anyway? The Role of the Teacher
Data
Results
Let’s Do This!

3. Learning Analytics…?
Learning Analytics (LA) has been defined as the “measurement, collection, analysis, and reporting of data about learners and their contexts, for the purposes of understanding and optimizing learning and the environments in which it occurs” (Siemens 2013).
What is still unclear: how are we to collect, manage, interpret and use this data (this depends on who you ask…)
In Practice, LA allows for data-driven continual improvement measures
Image:

4. Research is Outpacing Practice
The MAJORITY of research on the topic of Learning Analytics does not involve teachers. Rather, there is an abundance of high level and computer science-based work.
We’re missing out on data gathering on the qualitative side of the learning experience. And we’re missing out on what teachers can do with LA right now.

5. Data-Informed Decision Support
Teacher Inquiry: Teachers engage in reflective activities to strengthen their practice, and share their experiences with peers.
Learning Design (LD): Sharing tried- and-true methods of design principles in teaching and learning with technology.

6. Methodology Study Design Mixed methods exploratory case study
Sample Multiple iterations of two established online graduate courses taught by long-standing, highly experienced faculty.

7. Methodology Step 1: Quantitative Analysis of LMS data
→ Extract LMS data in the form available to faculty
→ Analyze according to:
Learning Design Visualizations
As described in Lockyer et al, 2013 and supported in Bakharia, Corrin, de Barba, Kennedy, Gašević, Mulder, Williams, Dawson and Lockyer (2016)
Total log counts (as proxy for total presence)
See Whitmer’s study (as cited in Jisc, 2016)
Discourse Analysis using the Community of Inquiry (CoI) framework
See analytics models: Rockinson-Szapkiw, Wendt, Whighting & Nisbet (2016), and original framework by Garrison, Anderson & Archer (1999)

8. Methodology, continued
Step 2: Qualitative Analysis – insight from faculty
LMS data representations were provided to faculty with no explanations, then an open-ended interview was conducted
Step 3: Compare/contrast findings from the data and insights from faculty.
Goal: Look for practical implications emerging from this process

9. Results Important highlights:
Faculty believe that their regular use of the crude LMS data is beneficial to their practice for both monitoring and reflection
Alignment between learning design and user activity was clearly evident
Variations in user activity (presence) was explained by faculty in surprising ways. Most notable was flexibility provided to students which was vital to the success of some, and various technology and communication preferences that caused uneven data capture.
The LMS captured only a fraction of learning activities

10. Results
There’s more!
A rich CoI was evident with high levels of the three presences (cognitive, social and teaching).
The dynamics (establishment and timing) of the three presences were consistently unique in each course.
Faculty insight made it clear that CoI differences were influenced by the teacher’s leadership approach.
The visualizations provided to faculty were not well understood. Each faculty had their own ideas about what they wanted to see, supporting a call for on-demand analytics.

11. Let’s make a Change! Next moves…
Gašević, Dawson, Rogers and Gasevic (2016) study the extent to which instructional conditions influence predictions of academic success and caution that generalized models miss this important parameter.
Next moves…
Give teachers the resources to:
inform their practice
share their experience
Remove barriers by providing:
Technology support
Academic freedom
Time and incentive to foster a community of practice
Image:

12. A Few References..
Bakharia, A., Corrin, L., de Barba, P., Kennedy, G., Gašević, D., Mulder, R., Williams, D., Dawson, S. & Lockyer, L. (2016). A conceptual framework linking learning design with learning analytics. LAK ’16 Conference Proceedings: The Sixth International Learning Analytics & Knowledge Conference, pp
Garrison, D. R., Anderson, T., & Archer, W. (1999). Critical inquiry in text-based environments: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87–105.
Gašević, D., Dawson, S., Rogers, T. & Gasevic, D. (2016). Learning analytics should not promote one size fits all: the effects of instructional conditions in predicting academic success. The Internet and Higher Education 28, 68– 84.
Kovanović, V., Gašević, D., Joksimović, S., Hatala, M. & Adesope, O. (2015). Analytics of communities of inquiry: Effects of learning technology use on cognitive presence in asynchronous online discussions. Internet & Higher Education, (27), doi: /j.iheduc
Persico, D., & Pozzi, F. (2015). Informing learning design with learning analytics to improve teacher inquiry. British Journal of Educational Technology, 46(2), doi: /bjet.12207
Rockinson-Szapkiw, A., Wendt, J., Whighting, M. & Nisbet, D. (2016). The Predictive Relationship Among the Community of Inquiry Framework, Perceived Learning and Online, and Graduate Students’ Course Grades in Online Synchronous and Asynchronous Courses. The International Review of Research in Open and Distributed Learning, 17(3).

14. Why the CoI?
In the context of graduate level online learning, a collaborative-constructivist model applies, which is the basis of the Community of Inquiry (CoI) Model (Garrison, Anderson and Archer, 1999).
Supplementary slide in case of audience query RE: CoI
Validation support of the CoI model notes the dual nature of the element of teaching presence, with course design and instructor behaviour emerging as separate and related factors
Image: