1. Jozef Tvarožek and Mária Bieliková Enhancing Learning with Off-Task Social Dialogues EC-TEL 2010, Barcelona September 30, 2010 Slovak University of Technology in Bratislava

2. Our Approach to Socially Intelligent Tutor Enhancing Learning with Off-Task Social Dialogues2

3. Learning (1) – Course notes Enhancing Learning with Off-Task Social Dialogues3

4. Learning (2) – Problem solving Enhancing Learning with Off-Task Social Dialogues4

5. Tasks for assessment and practice Expert’s idea Task scheme Task instance Answer Answer category Ability estimate Adaptive selection Generator StudentJudge Estimation 5Enhancing Learning with Off-Task Social Dialogues

6. Task scheme specification A B C A - C A - B Solution tree B - C Task’s scheme tree Parameters, constraints, tree of subtasks and answers Psychometric IRT parameters, usage indicators Task scheme: correct incorrect 6Enhancing Learning with Off-Task Social Dialogues

7. Task instance Parameters’ specification Instantiated parameters Scheme tree Instance tree Pruned backtracking Combine Instance generation 7Enhancing Learning with Off-Task Social Dialogues

8. Updating user’s profile Off-task dialogues – Qs/As scripted to perform actions Extracting user’s preferences & behaviors Extracting event attributes Recommending events to attend Negotiating events with others – Relationship maintenance 8Enhancing Learning with Off-Task Social Dialogues

9. Extracting interests Enhancing Learning with Off-Task Social Dialogues9 Tutor: now, look around and solve exercises, ok? see you around! Student ack / Turn initiative Tutor: interesting, I for example like to read books, swim, play volleyball and soccer ≥ 40 chars < 40 chars Extract features (e.g. to draw, watch TV, friends) Tutor: Hello Kate, how are you? I'm here to make you feel comfortable, so that you learn much... :-) Tutor: ok, write me about yourself, what you like, and all... I can then prepare exercises that you will like... ;) Student greeting Tutor: tell me more, pls.

10. Sample conversation Enhancing Learning with Off-Task Social Dialogues10 Joe: Hello Kate, how are you? I’m here to make you feel comfortable, so that you learn much… : -) Kate:thnx Joe: ok, write me about yourself, what you like, and all… I can then prepare some exercises that you will definitely like : P Kate:like to draw sleep watch TV … Joe:write a bit more please … Kate:like to go out go out with my dog Joe: interesting, I for example like to read books, from sports volleyball, swimming, and also some soccer Kate:okay Joe: now, look around this environment and solve some of the exercises, ok? see you around!

11. Real-life adaptation of tasks Parameters’ specification Instantiated parameters Scheme tree Instance tree Pruned backtracking Combine Instance generation, guided by student’s hobbies 11Enhancing Learning with Off-Task Social Dialogues Semantic similarity with student’s favorite concepts

12. Evaluation study Middle school mathematics 18 parametric algebra tasks Tutoring friend – Extract hobbies Students did participate in a pilot previously – Familiar with the environment Enhancing Learning with Off-Task Social Dialogues12

13. Is it better than paper&pencil? 32 students – Control group = traditional classroom – Experimental group = tutor – Learning gain: 1.2% vs 10.3% Enhancing Learning with Off-Task Social Dialogues13 pre-testpost-testgaint-test meanst.devmeanst.devmeanst.devt statp-value Control group0.6970.2300.7090.2110.0120.258-0.180.428 Exp. group0.4340.2530.5380.2690.1030.172-2.400.015

14. Are they willing to do it? 16 students Detect student interests in the initial welcome dialogue: to draw, sleep, watch TV, go out, go out with dog Mean word count 11.6 (st.dev 8.7) Mean feature count 1.56 (st.dev 1.7) 44% IGNORED the tutor – Others: mfc 2.78 (st.dev 1.39) Enhancing Learning with Off-Task Social Dialogues14

15. Motivating students? Enhancing Learning with Off-Task Social Dialogues15 Those that did engage with the tutor – Less problems attempted, higher success rate. ControlExperimental meanst.devMeanst.dev Number of tasks attempted7.713.867.001.80 Number of tasks solved correctly2.851.464.002.45 Questions (response scale: 1=worst, 5=best) 1. How much did you learn in the tutor?2.291.253.330.71 2. How much did the tutor help you on the post-test? 2.291.113.331.12 3. How much would you like to use the tutor again? 3.141.074.331.12 4. How did you like the tutor?2.861.074.220.97

16. Motivating students? (contd.) Enhancing Learning with Off-Task Social Dialogues16 Is the tutoring friend any good? – We don’t know. – Learning gain: 3.7% vs. 12.3% – We can filter students that are engaged, and do well. pre-testpost-testgain meanst.devmeanst.devmeanst.dev Engaged0.4290.2450.4650.2830.0370.283 Not engaged0.4390.2730.5620.2840.1230.192

17. Summing up Those who engage in the social off-task dialog with the tutor solve problems better :) Tutors that are “friends” with students can produce higher learning gains. Socially intelligent tutor – tutoring friend: – gets to know you better, – guides you to what you need. Enhancing Learning with Off-Task Social Dialogues17

18. Jozef Tvarožek and Mária Bieliková Enhancing Learning with Off-Task Social Dialogues EC-TEL 2010, Barcelona September 30, 2010 Slovak University of Technology in Bratislava