711: Intelligent Tutoring Systems Week 1 – Introduction

2 Today  Introductions  Some overviews… Course objectives and course structure Syllabus and assignments Intelligent tutoring systems  Reading discussion Why ITSs? What this course will cover and what it won’t cover  Teams and topics  Create your first interface and behavior graph

3 Introductions  Your name  Department  Research interest  What’s your motivation for taking this course?  What do you expect to learn?

4 OVERVIEWS

5 Course objectives  Relate learning sciences theory to practice You’ll learn about research-based principles about how to design effective instruction You’ll create an ITS that makes use of these principles  Work in a domain of your choice Think about how the assignments can be useful within your own discipline of interest Think about how you can make the assignments useful for your own research

6 Course structure  Part I (between classes): Read about some research-based principles about learning and instruction Reflect and plan how you would apply these principles in designing your own ITS (design posts)  Part II (in class): Critical discussion of principles  Part III (in class and at home): Putting the principles into practice Work on your ITS in class Finish what you did not complete in class at home  Part IV (final): Your own ITS!

7 Syllabus…

8 Assignments  Always: Readings Design posts Comment on others’ design posts Finish implementations that you did not complete in class  5/6: Final presentations Final write-ups

9 INTRODUCTION TO INTELLIGENT TUTORING SYSTEMS (my only “lecture” this semester)

10 Outline  What is ‘a tutor?’ Some examples  Use of CTAT to author tutors Motivation Common features: What CTAT can do Demos Examples of projects that have used CTAT  Evidence of authoring efficiency with CTAT

11 President Obama on ITSs “[W]e will devote more than three percent of our GDP to research and development. …. Just think what this will allow us to accomplish: solar cells as cheap as paint, and green buildings that produce all of the energy they consume; learning software as effective as a personal tutor; prosthetics so advanced that you could play the piano again; an expansion of the frontiers of human knowledge about ourselves and world the around us. We can do this.” blin/gGxW3n

12 Cognitive Tutor Algebra Use graphs, graphics calculator Analyze real world problem scenarios Use table, spreadsheet Use equations, symbolic calculator Tracked by knowledge tracing Model tracing to provide context-sensitive Instruction

13 Cognitive Tutor Geometry

14 The nested loop of conventional teaching For each chapter in curriculum  Read chapter  For each exercise, solve it  Teacher gives feedback on all solutions at once  Take a test on chapter VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3),

15 Nested loops of computer-assisted instruction For each chapter in curriculum  Read chapter  For each exercise Attempt answer Get feedback & hints on answer; try again If mastery is reached, exit loop  Take a test on chapter VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3),

16 The nested loops of ITSs For each chapter in curriculum  Read chapter  For each exercise For each step in solution o Student attempts step o Get feedback & hints on step; try again If mastery is reached, exit loop  Take a test on chapter VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3),

17 Common Tutor Features 1. Problem-type-specific interfaces with multiple representational tools, designed to make thinking steps visible (Anderson, Corbett, Koedinger, & Pelletier, 1995); 2. Correctness feedback on problem-solving steps 3. Feedback messages for commonly-occurring errors; 4. On-demand hints about what to do next, available at any point during problem solving; 5. Background materials such as a Glossary, online textbook, worked examples, etc. 6. An open learner model (dubbed “skill meter”) that displays the system’s estimates of an individual student’s skill mastery 7. Individualized problem selection based on each student’s performance with the tutor

18 Design/Development Process  As important as the “tutor features” is grounding the design (and re-design) process in a strong understanding of student thinking of learning: Use of cognitive task analysis up front to inform initial design Use of student data (e.g., tutor log data in the DataShop or results from in vivo studies) to inform redesign

19 Demo 1  Fractions Tutor

20 Feedback Studies in LISP Tutor (Corbett & Anderson, 1991) Time to Complete Programming Problems in LISP Tutor Immediate Feedback Vs Student-Controlled Feedback

21 What does it mean to say that tutors are “adaptive?”

22 Demo 2  ChemTutor

23 CTAT : easier and faster tutor development!  Cognitive Tutors: Large student learning gains as a result of detailed cognitive modeling ~200 dev hours per hour of instruction (Koedinger et al., 1997) Requires PhD level cog scientists and AI programmers  Development costs of instructional technology are, in general, quite high E.g., ~300 dev hours per hour of instruction for Computer Aided Instruction (Murray, 1999)  Solution: Easy to use Cognitive Tutor Authoring Tools (CTAT) Murray, T. (1999). Authoring Intelligent Tutoring Systems: An Analysis of the state of the art. The International Journal of Artificial Intelligence in Education, 10, Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. The International Journal of Artificial Intelligence in Education, 8,

25 Tutors supported by CTAT  Cognitive Tutors Difficult to build; for programmers Uses rule-based cognitive model to guide students General for a class of problems  Example-Tracing Tutors Novel ITS technology Much easier to build; for non-programmers Use generalized examples to guide students Programming by demonstration One problem (or so) at a time

26 Building an example-tracing tutor 1. Decide on educational objectives 2. Cognitive Task Analysis 3. Design and create a user interface for the tutor 4. Demonstrate correct and incorrect behavior (i.e., create a behavior graph) Alternative strategies, anticipated errors 5. Generalize and annotate the behavior graph Add formulas, ordering constraints Add hints and error messages Label steps with knowledge components 6. Test the tutor 7. (Optional) Use template-based Mass Production to create (near)- isomorphic behavior graphs 8. Deliver on the web - import problem set into LMS (TutorShop)

27 CTAT’s utility  Over 500 CTAT users in summer schools, courses, workshops, research, and tutor development projects Domains: mathematics, chemistry, genetics, French culture, Chinese, ESL, thermodynamics At least 44 research studies built tutors and deployed in real educational settings  In the past two years CTAT was downloaded 6,600 times the CTAT website drew over 2.9M hits from 164k unique visitors

28 Other CTAT topics  Techniques to make example-tracing tutors flexible Multiple paths, formulas, ordering constraints  Techniques to batch-author isomorphic and near-isomorphic problems Mass Production combined Excel  Evidence of authoring efficiency for example- tracing tutors

29 READING DISCUSSION

30 Why Intelligent Tutoring Systems?

… will cover Model tracing Knowledge components Mass production Learning sciences principles for instructional design … will not cover Knowledge tracing Production rules DataShop and tutor log data analysis Empirical evaluations What this course…

32 YOUR FIRST OWN ITS

33 Individual or Group Work  Come up with 3 tasks that could be implemented in a tutoring system  Tutor problems should: Include multiple steps Lend themselves to creating multiple versions that increase in difficulty  Sketch out your ideas on paper

34 Practice Session Flash  Create a simple interface for one of your problems  Use at least the following components: CommShell CommTextInput Done button

35 Practice Session CTAT  Create a simple behavior graph for one of your problems  Steps: Create start state Demonstrate solutions Annotate solution steps: o Incorrect steps o Hint messages  Test the tutor

36 FOR NEXT WEEK

37 Tasks before next week  Finish interface and behavior graph for your problems  Do the assigned readings  Post on Moodle (Sunday, 11:59pm) how you plan to conduct an expert cognitive task analysis, apply the subgoaling principle to your tutor problem, and how you plan to come up with a sequence of tutor problems for your task  Comment on others’ posts