1. Open-source LA and “what the student does” Dr Phill Dawson, Monash Dr Tom Apperley, Melbourne

2. Structure Until 3:30pm You will get a break I will talk a bit (background concepts) I will show some tools You will talk a lot You write an algorithm You will probably argue about ethics

4. Dr Phillip (Phill) Dawson Lecturer in Learning and Teaching at Monash Led a small grant in learning analytics Interested in how academics make decisions

5. Who are you? LA researchers? Educational designers? LMS administrators? University managers? Faculty-based academics? Academic developers? Non-university?

6. “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” (SoLAR)

7. What LA do you use currently in everyday teaching? (ie not research) Nothing? Reports? (eg ‘who has logged in?’; ‘who has submitted assignment one?’) Dashboards? Something else?

8. “Who is struggling or not engaging with my course?”

10. Free, modular, configurable extendable, open-source learning analytics block for teachers to identify students at risk

11. “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” (SoLAR)

12. Learning happens because of… 1.What the student is? 2.What the teacher does? 3.What the student does? Biggs 1999

13. “learning is what the student does”

14. Activity - pairs What do students do to learn in your context? – Long list – Specific – Verb stems – Online and offline – Effective and ineffective – Deep and strategic

15. Which of these can easily be captured by LA? Which of these can’t possibly be captured by LA?

16. Sci-fi LA vs Real LA Flickr user sndrv http://www.flickr.com/photos/sndrv/4519088620/ CC-BYhttp://www.flickr.com/photos/sndrv/4519088620/ Flickr user dpape http://www.flickr.com/photos/dpape/2720632752/ CC-BYhttp://www.flickr.com/photos/dpape/2720632752/

17. Typical Open-Source LA Tools Gather data on student use of parts of LMS No integration with Student Management Systems Teacher dashboards Reports Some synthesis Inconsistent design and language – At code + UI levels

18. State of the Actual: Free/open/built-in LA Open-source tools – Engagement Analytics (documentation, demo vid)documentationdemo vid – Gismo Gismo – Analytics and Recommendations Analytics and Recommendations Vendor-supplied reports – (eg Desire2Learn)Desire2Learn

21. Modular ‘indicator’ architecture (so you can make additional indicator plugins) Attendance Downloads Completio n Facebook

22. If an assignment is very late it is riskier than if it is just a little late If an assignment is worth a greater percentage then it is riskier than if it is worth a small percentage If an assignment is past its due date and not submitted then it is riskier than if it was submitted late How does the assessment indicator work?

23. for each assignment, quiz, lesson in the course whose due date has passed { daysLateWeighting = ((number of days late) - overdueGraceDays) / (overdueMaximumDays - overdueGraceDays) assessmentValueWeighting = (value of this task) / totalAssessmentValue if (daysLateWeighting > 1) { daysLateWeighting = 1 } else if (daysLateWeighting < 0) { daysLateWeighting = 0 } if (task was submitted late) { risk = risk + daysLateWeighting * assessmentValueWeighting * overdueSubmittedWeighting } else if (task was not submitted) { risk = risk + daysLateWeighting * assessmentValueWeighting * overdueNotSubmittedWeighting }

24. Activity: specify a new ‘indicator’ of learning in your context What it does in one sentence Procedure to give a number between 0% (no risk) and 100% (high risk) – Words? – Pictures? – Flowchart? – Algorithm? What variables could we tweak? How important is this indicator?

25. Ramsden 1992, p. 187 “From our students’ point of view, the assessment always defines the actual curriculum”

26. “Students can, with difficulty, escape from the effects of poor teaching…” Boud 1995, p. 35

28. “…they cannot (by definition if they want to graduate) escape the effects of poor assessment” Boud 1995, p. 35

29. Teaching/LearningAssessmentObjectives/Outcomes

31. LA are only as good as curriculum Training statistical LA on assessment or retention outcomes ≠ learning Teacher needs to be in control of LA use and specify learning LA makes it more difficult to “escape the effects of bad teaching”

32. “whether through denial, pride, or ignorance, students who need help the most are least likely to request it” Martin & Arendale 1993 p. 2

33. It’s the end of week 2 and student X hasn’t ever logged in. What do we do?

34. How can we make follow-up effective? Personal or robotic? Paint a grim picture? Refer on or see personally? Specific guidance

35. It’s the week of the census and modeling suggests student X is 70% likely to fail. What do we do?

36. How can we make follow-up ethical? Do students have a – Right to try (and fail?) – Right to give up – Right to be strategic Will draconian measures lead to LMS-farming? Student-identified triggers

37. It’s week 10 and student X already has 60% of the course grade but hasn’t logged in for two weeks. What do we do?

38. Discussion and close: the near-future for open-source learning analytics

39. Extra time options Live demonstration of Engagement Analytics tool Live demonstration of Engagement Analytics tool Further specifying an indicator into pseudocode for a developer Develop strategies for following up students at risk Discuss student views of analytics tools Discuss open-source

40. References and sources SoLAR definition of LA Memes are from Quickmeme NetSpot Innovation Fund logo courtesy of NetSpot. (You should apply for an open- source development grant through them.) NetSpot Innovation Fund Biggs, J. (1999). What the Student Does: teaching for enhanced learning. Higher Education Research & Development, 18(1), 57-75. doi: 10.1080/0729436990180105 Boud, D. (1995). Assessment and learning: contradictory or complementary. In P. Knight (Ed.), Assessment for Learning in Higher Education (pp. 35-48). London: Kogan Page. Ramsden, P. (1992). Learning to teach in higher education. London: Routledge. Martin, D., & Arendale, D. (1993). Supplemental Instruction: Improving First-Year Student Success in High-Risk Courses The Freshman Year Experience: Monograph Series (2nd ed., Vol. 7). Columbia, SC: National Resource Center for the First Year Experience and Students in Transition, University of South Carolina.