Integrated Collaborative Learning Environments with Dynamic Support Carolyn Penstein Rosé Language Technologies Institute/ Human-Computer Interaction Institute

Design Principle Know what problem you are trying to solve!!

Passivity in Class Discussions

Accountable Talk A codified set of moves that Ts and Ss can be taught that build effective language in the classroom Press individuals to explain and elaborate Require individuals to listen and respond to each other Reveal misunderstandings, incomplete ideas.. Let Ss know how others are “hearing them” Spread participation through the group

Transactivity Students explicitly display their reasoning Students orient their contributions towards previous contributions  May be competitive or non-competitive  May be representational operational  May be oriented towards self or other Student 1: Alright, we put that the negative and positive whole numbers. Teacher: Alright, other ideas you want to add on, yes Mario. Student 2: I put a um numbers that doesn't include decimal points or fractions but they include, they can include negative signs or positive signs.

Accountable Talk MoveCorresponding Transactivity Move RevoicingClarification, Competitive Clarification, Refinement Restate RequestParaphrase, Competitive Paraphrase Reasoning Application Request Extension, Competitive Extension, Contradiction, Reasoning Critique Prompting for Further Participation. Extension, Completion, Clarification Request for ElaborationExtension, Completion Challenge or Request for Counter-Example Competitive Paraphrase, Contradiction, Competitive Extension Accountable Talk elicits Transactivity in group discussion

Eddie: Well, i don't think it matters what order the numbers are in. You still get the same answer. But three times four and four times three seem like they could be talking about different things. Teacher: Rebecca, do you agree or disagree with what Eddie is saying? Rebecca: Well, I agree that it doesn't matter which number is first, because they both give you twelve. But I don't get what Eddie means about them saying different things. Teacher: Eddie, would you explain what you mean? Eddie: Well, I just think that like three times four can mean three groups of four things, like three bags of four apples. And four times three means four bags of three apples, and those don't seem like the same thing. Tiffany: But you still have the same number of apples, so they are the same! Teacher: OK, so we have two different ideas here to talk about. Eddie says the order does matter, because the two orders can be used to describe different situations. So Tiffany, are you saying that three times four and four times three can't be used to describe two different situations? Reasoning Application Request Justification Request Request for Elaboration Clarification Reasoning Critique Revoicing

Unequal Participation

Wizard-of-Oz Setup Tutor Student

Problem Solving Interface Interface allows students to submit Incorrect solutions

Reflection Interface

Scaffolding for Collaboration Prompts to encourage the instructionally beneficial behavior of students –Explanations –Equal contributions

Explanation Prompt (Student 1 corrects the answer for the confederate learner without offering any explanation) Student1: c is 1 Auto-Prompt (to student 1): Help the student understand your correction. Student2 : what does c mean? Student1 : constant Student2 : ok Student1 : the constant in front of the variable which in our case is variable t Student2 : ok Student1 : now for the second part we use the constant function

Equal Contribution Prompt (After the participant makes 5 correct problem actions without confederate student’s contribution) Auto-Prompt: It seems like the other student has not contributed lately. Why don’t you see if they need help? Student1: you want to do the other half? Student2: ok

Learning effect Significant benefit for prompting on student learning with pretest as a covariate  F(1,39) = 4.47, p <.05, effect size.58

Process Losses

Another Domain: Earth Sciences Collaborative idea generation in the Earth Sciences domain Chinese TagHelper learns hand-coded topic analysis  Human agreement.84 Kappa  TagHelper performance.7 Kappa Trained models used in follow-up study to trigger interventions and facilitate data analysis

Example Dialogue SpeakerText Student 1 People stole sand and stones to use for construction. VIBRANT Yes, steeling sand and stones may destroy the balance and thus make mountain areas unstable. Thinking about development of mountain areas, can you think of a kind of development that may cause a problem? Student 2 Development of mountain areas often causes problems. Student 1 It is okay to develop, but there must be some constraints. * Feedback during idea generation increases learning and idea generation productivity (except during the first 5 minutes) (Wang et al., 2007) Pairs+ Feedback Individuals+ NoFeedback Pairs+ NoFeedback Individuals+ Feedback

Process Analysis Process loss Pairs vs Individuals: F(1,24)=12.22, p<.005, 1 sigma Process loss Pairs vs Individuals: F(1,24)=4.61, p<.05,.61 sigma Negative effect of Feedback: F(1,24)= 7.23, p<.05, sigma Positive effect of feedback: F(1,24)=16.43, p<.0005, 1.37 sigma Pairs+Feedback Individuals+NoFeedback Pairs+NoFeedback Individuals+Feedback Pairs+ Feedback Individuals+ NoFeedback Pairs+ NoFeedback Individuals+ Feedback

Deficient Help Exchange

Why study help? Offering of deep help and elaborated explanations predicts post test performance (e.g., Webb et al., 2002) Help behavior mediates learning (e.g., Gweon et al., 2006; Gweon et al., 2007) Exchanging help in the context of collaborative learning increases identification with a learning community and motivation, and improves race relations (Sharan, 1980) Providing help leads to feelings empowerment for “low status” students (Elbers & Hann, 2004)

How can we prompt helping behavior? DIRECT: Explicit prompts (Gweon et al., 2006) LESS DIRECT: Manipulating availability of help from problem solving environment (Gweon et al., 2007)  Girls offered more help with delayed feedback from environment  Boys offered more help with immediate help from the environment SUBTLE: Conversation agents inject humor (Kumar et al., 2007)  Positive effects on perceived help exchange (p<.05), effective help exchange (p<.1), and Learning (p=.06)

Collaborative Problem Solving Environment Jan packed several books to amuse herself on along car ride to visit her grandma. After 1/5 of the trip she had already finished 6/8 of the books she brought. How many times more books should she have brought than what she packed?

Extraneous Entertainment?

Coding Scheme (Gweon et al., 2007) Help provision mediates learning (Gweon et al., 2006; Gweon et al., 2007) (R) Help Requests: “Help me”, “I’m stuck”, “I don’t get it.” (P) Help Provisions: “Find the common denominator”, “Try the flip and multiply strategy” (C) Can’t help: “I don’t know”, “I’m stuck too” (D) Deny help: “ask the teacher”, “you’re an idiot”, “press the help button” (N) Nothing

Examples [R] Student 1: What do we put on top of the fraction? [P] Student 2: Did you find a common denominator? [R] Student 1: I don’t get it [D] Student 2: hold on [R] Student 1: Why 16? [C] Student 2: I don’t know. [R] Student 1: I don’t get it

Big findings: More help related episodes in experimental condition More episodes where people got help, and then managed to solve the problem themselves in the experimental condition

Vision

Support for collaborative learning is like training wheels Effective support allows learners to achieve better collaboration Unnecessary support can be demotivating Fading support is ideal But too little support can be detrimental as well Trained human facilitators are able to achieve the right balance

Vision Support for collaborative learning is like training wheels Effective support allows learners to achieve better collaboration Unnecessary support can be demotivating Fading support is ideal But too little support can be detrimental as well Trained human facilitators are able to achieve the right balance

Vision Support for collaborative learning is like training wheels Effective support allows learners to achieve better collaboration Unnecessary support can be demotivating Fading support is ideal But too little support can be detrimental as well Trained human facilitators are able to achieve the right balance

Vision Support for collaborative learning is like training wheels Effective support allows learners to achieve better collaboration Unnecessary support can be demotivating Fading support is ideal But too little support can be detrimental as well Trained human facilitators are able to achieve the right balance

Vision Support for collaborative learning is like training wheels Effective support allows learners to achieve better collaboration Unnecessary support can be demotivating Fading support is ideal But too little support can be detrimental as well Trained human facilitators are able to achieve the right balance

Support for collaborative learning is like training wheels Effective support allows learners to achieve better collaboration Unnecessary support can be demotivating Fading support is ideal But too little support can be detrimental as well Trained human facilitators are able to achieve the right balance Vision

Architecture for Dynamic Collaboration Support

VMT-Basilica ** Students learn up to 1.25 standard deviations more when interactive support is provided in the environment.

Basilica Architecture A channel filter collects all of the events that occur at the interface from all students involved in the conversation

Basilica Architecture Events are then channeled to special purpose filters such as a text processing filter Other filters may keep track of time or other factors not related to student behavior

Basilica Architecture Filters that are related to decision making for specific types of interventions receive notifications from the analysis filters and pass notifications on to the Actors, which are responsible to launching interventions

Basilica Architecture Actors of different types produce events that are sent to the Outgoing Message Spooling Filter From there, events are sent through the Presence Actor to the interface.

Issues to consider What problem are you trying to solve?  Formulate analysis scheme When should you intervene?  Use technologies like TagHelper and SIDE to track interaction and trigger support What should the intervention be?  Technologies like TuTalk can be used to offer support

Questions?