1. Simply Complicated: The Use of Student Interaction Data in Online Learning Environments to Inform Teacher Inquiry and Learning Design A report on the literature
Presented by Marie Lippens
Candidate,
Master of Education in Distance Education
Athabasca University

2. Outline
Context in Online Learning Learning Analytics Teacher Inquiry Learning Design Current Research Moving On

3. Why Are Online Educational Systems Difficult to Characterize?
They are uniquely placed in a dynamic social system
Institutions
Learners
Teachers
Online Learning
Technology
Culture
Economy
Employers
Politics

4. Big Data
Learning Analytics (LA) and Educational Data Mining (EDM) are two related fields concerned with understanding behaviours in online teaching and learning.
Still unclear: how are we to collect, manage, interpret and use this data (this depends on who you ask…)
Overarching goal: data-driven continual improvement measures to keep up with a changing landscape.

5. Data-Informed Decision Support
Teacher Inquiry: teachers share their experiences with peers and engage in reflective activities to strengthen their practice.
Learning Design (LD): represents new ways to share tried-and-true methods of design principles in teaching and learning with technology.

6. Student Activities and Success
In the context of graduate level online learning, a collaborative-constructivist model applies, which is the basis of the Community of Inquiry (CoI) Model (Garrison, Anderson and Archer, 1999).
Validation support of the CoI model notes the dual nature of the element of teaching presence, with course design and instructor behaviour emerging as separate and related factors
Image:

7. In a Particular Context…
Interaction data collected within online courses can be used to assess alignment of course design parameters, course delivery elements and learning outcomes.
Example:
Park, Yu and Jo (2016) categorized hybrid courses based on planned activities. A modified categorization applies to fully online courses:
C: Communication or collaboration
D: Delivery or discussion
S: Sharing or Submission
Student activity, quality metrics and/or prediction models should be very different for each category

8. Examples of LA applications…
Prediction
Macfadyen and Dawson (2010) produced a highly accurate predictive model of student success using LMS data analysis to mine predictive variables.
Zhou and Winne (2012) looked at LMS trace data representative of mental events (tagging text, clicking on hyperlinks), and found that the data was more predictive of goal orientation and student success when compared to students’ self-reported goals.
Kim, Park, Yoon and Jo (2016) looked at the use of proxy variables mined from student discussion forum data, and a prediction model of success was developed and validated. The model showed impressive accuracy of 70% in week 1 of the course. This figure rose to greater than 90% by the middle of the course. This level of accuracy was achieved even though the discussion forum analyzed only accounted for 15% of the course final grade.
Rockinson-Szapkiw, Wendt, Whighting and Nisbet (2016) provide strong support for the foundational constructs of Community of Inquiry (CoI) theory and the role of perceived learning while using learning analytics to predict course points.

9. Examples of LA applications…
Teacher use of LD as context for LA
Kim and Lee (2012) created a tool that looks at student interactions with a multidimensional approach in order to allow for a flexible analytical framework that teachers and researchers can customize to their context.
Lockyer, Heathcote & Dawson (2013) support using LD to document pedagogical intent, providing context for data interpretation.
Persico and Pozzi (2015) consider the use of LA to transform LD from a craft into a data-grounded research area, supporting enquiry while teachers design, run and evaluate the learning process.
Rodríguez-Triana, Martínez-Monés, Asensio-Pérez and Dimitriadis (2015) performed a design-based LA project that asked teachers to identify their design approach and analytics needs with great success at informing future course improvements.
“Craft”: based on experience, intuition and tacit knowledge

10. Building on Current Studies
Gašević, Dawson, Rogers and Gasevic (2016) study the extent to which instructional conditions influence predictions of academic success and caution that generalized models miss this important parameter.
Next moves…
Give teachers the resources to:
inform their practice
share their experience
Remove barriers by providing:
Technology support
Academic freedom
Time and incentive to foster a community of practice

11. …Not this

12. A Few References..
Garrison, D. R., Anderson, T., & Archer, W. (1999). Critical inquiry in text-based environments: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87–105.
Gašević, D., Dawson, S., Rogers, T. & Gasevic, D. (2016). Learning analytics should not promote one size fits all: the effects of instructional conditions in predicting academic success. The Internet and Higher Education 28, 68– 84.
Kim, D., Park, Y., Yoon, M. & Jo, I. (2016). Toward evidence-based learning analytics: Using proxy variables to improve asynchronous online discussion environments. The Internet and Higher Education, 30,
Park, Y., Yu, J. H. & Jo, I. (2016). Clustering blended learning courses by online behavior data: A case study in a Korean higher education institute. The Internet and Higher Education, 29, 1-11.
Persico, D., & Pozzi, F. (2015). Informing learning design with learning analytics to improve teacher inquiry. British Journal of Educational Technology, 46(2), doi: /bjet.12207
Rockinson-Szapkiw, A., Wendt, J., Whighting, M. & Nisbet, D. (2016). The Predictive Relationship Among the Community of Inquiry Framework, Perceived Learning and Online, and Graduate Students’ Course Grades in Online Synchronous and Asynchronous Courses. The International Review of Research in Open and Distributed Learning, 17(3).
Rodríguez-Triana, M. J., Martínez-Monés, A., Asensio-Pérez, J. I. & Dimitriadis, Y. (2015). Scripting and monitoring meet each other: Aligning learning analytics and learning design to support teachers in orchestrating CSCL situations. British Journal of Educational Technology, 46(2), doi: /bjet.12198