Engaging with Massive Online Courses Ashton Anderson, Jure Leskovec Stanford University Daniel Huttenlocher, Jon kleinberg Cornell University

MOOC introduction Massive open online course MOOC platforms :Coursera Some means: ML1..3and PGM1..3

Main points Taxonomy of engagement styles base on behavior Investigate forum relates to participate Large-scale deployment of badges

Taxonomy of engagement styles Fundament activities Viewing lecture Handing in an assignment for credit （ additional activities :upgraded quizzes and forum participation ） Natural styles of engagement Viewers.primarily watch lectures,handing few assignment Solvers. primarily hand in assignment viewing few All-rounders. Balance between Collectors. main download Bystanders.registered for course below low threshod.

Taxonomy of engagement styles Problems: 1. Not sharp boundaries a ： activity of assignment l: lectures 2. Engagement and Grades Most students receive zero Different course between ML and PGM(do all course work More challenging)

Taxonomy of engagement styles Define thresholds c0>=1 and 0<Q0<Q1<1 a Bystander if (a + l) <c0; otherwise, they are a Viewer or Collector if a/(a + l) <Q0; depending on whether they primarily viewed or downloaded lectures, respectively, an All-rounder if Q0 < a/(a + l) < Q1; a Solver if a/(a + l) >=Q1.

Time of interaction --correlate of their behavior Archaeologists ： first action in the class is after the end date of the class

Grades and student engagement overall final grade distributionin the two classes

Grades and student engagement Median number of actions of students with a given final grade in PGM2 and ML2.

Composition of near-perfect students Solver, who perhaps know the material, All-rounder, who watch lecture finish quiz assign

Course forum activity Question Which types of students visit the forums?

Course forum activity Develop an analysis framework can clarify how the forums be used. 1.Clarify the forum conversational structure 2.The thread form of participation 3.How Stronger and weaker student interact 4.Identify feature based on the context of post

Course forum activity If this number is close to k, it means that many students are contributing if it is a constant or a slowly growing function of k, then a smaller set of students are contributing repeatedly to the thread.

Course forum activity

Estimate a student’s eventual activity level from their forum post. All forum post for the first two weeks the course. Estimate words (W)

A large-scale badge experiment Two large-scale interventions(ML3) Design and implement badge system Run randomized experiment that presentation Of badge.

badge system Milestone badge :user win badge once they perform amount of activity. Badge types: bronze,silver,gold and diamond Award badge types : some actions (cumulative badge), authoring post or thread(accumulative great achievement) One time badges

badge system Effects of the badge system on forum engagement heavier tail—indicating that more users took more actions certain features of the distribution were stable prior to the striking difference exhibited by ML3, in which badges were offered. Didn’t show qualitatively significant differences in engagement between the three runs of the class.

Badge Presentation Experiment Question: How and why badge produce incentive effects. Do user view the badge as goal to be achieved for intrinsic personal reasons Were they viewed as signal social status

Badge Presentation Experiment Badge treatment conditions a)Top byline b)Thread byline a)Badge ladder

Badge Presentation Experiment Effect of badge treatment conditions. Badge-ladder clearly had the most significant effect. Top-byline and thread-byline were less significant but still performed better than we’d expect from null treatments

Conclusion Future work: 1.Predictive models of student behavior and grade. 2.Persinalization and recommendation mechanisms. 3.Further exploring badges. ……..

Thank you