1. The Behavior of Tutoring Systems
Kurt VanLehn
University of Pittsburgh

2. Underlying Claim
Behaviors of tutoring systems are quite similar, despite difference on
Task domain
User interface
Software structure
Knowledge base etc.

3. 6 Illustrative Tutoring Systems
Algebra Cognitive Tutor
Andes
AutoTutor
Sherlock
SQL-Tutor
Steve

4. Learning Event & Knowledge Component
Empirical tech. for establishing KC
Modeling transfer
Predicting learning curve
Determining grain size of KC
Instructive objective of designers
“the designers’ decisions about what learning events should be detected and advised by the tutor determine what knowledge components are included in the tutor’s representation of knowledge”

5. Two loops Outer loop Inner loop
select a task for student that will help to learn
Obtain a task set to select from
Inner loop
select the right step and help learning

6. I like this…
“a tutoring system does not have to replace a teacher or run an after-school remediation session. Its role in the student’s learning ecology can be anything—a smart piece of paper; an encouraging coach; a stimulating peer, etc” (p6-7)

7. The Outer Loop Task selection Student control
Homework helper e.g. Andes
Fixed sequence (random sequence)
e.g. AutoTutor, Assistments linear order curriculum
Mastery learning
Self-paced, e.g. Sherlock
Macro-adaptation
Knowledge component (KC) based. e.g. Cognitive Tutor
Systems modeling also incorrect KC (e.g.??)
Systems also represent learning styles and preferences (e.g. different pedagogical agents??)

8. The Inner Loop The goal is to help student to learn from a task
Common services:
Minimal feedback
Error-specific feedback
Hints on the next step.
Assessment of knowledge.
Review of the solution.

9. Minimal Feedback Categories When to give
Correct, incorrect, non-optimal, unrecognizable
When to give
Immediate
Delayed
Demand
Feedback policy can be a function of student competence (Razzaq & Heffernan, 2007)
Delayed feedback for high performing students
Immediate one for less competent students

10. Next-step Hints To avoid repeated guessing and frustration When Which
Based on student data (DT Tutor)
Hint on demand (Andes, Sherlock)
Hint abuse/refusal
“Proactive” help (DT Tutor, Steve)
Which
Following student’s plan
How
Single KC: Point, Teach, Bottom-out
Multi KC: bottom-out, refer to history; Scaffolding?
Contingent tutoring on student recent performance

11. Error-specific Feedback
Analyzing an incorrect step to determine the incorrect learning event and to prevent future occurrence
To help self-debugging
How
Minimal feedback + a sequence of hints
When
Immediate; when same error happens again; on demand etc.
Step analyzer seems a magic, black box to me. It seems it is impossible, at least very hard, to detect what KC a student applied in case of an incorrect answer.

12. Assessing Knowledge What kind of assessment is required?
Assessments are decision aids
Coarse-grained assessment
Combination of different measures
Fine-grained assessment
Based on learning events on steps
“Mastery” really means the probability that a knowledge component will be applied when it should be applied.
Assignment credit & blame problem
Counting learning events
Counting failures
Other issues
Here comes the step analyzer again: given a correct student step, it returns the set of knowledge components that were used to derive the step.
It is kind of easy to understand “blame” is a difficult problem. But why credit too? Is it possible for the tutor to credit different KCs in case of a correct step? E.g. given an external and an internal angles’ degree, asking for another one

13. Reviewing the Whole Solution
Real-time task domains
Non-real-time task domains
Pedagogical reason: delayed feedback
Plans and policies
Learn from human tutor (bottom-hint during performance, reasoning during reviewing)
Other issues

14. Final Remarks
The dream of having an ITS for every student in every course is within our grasp. 

15. Q? Is Assistment single-loop or double-loop?
Knowledge component can be incorrect (p3)? Is it saying the KC itself is wrong or not applied at proper step?
#steps per task is not fixed, varying by student;
Steps <-> learning events (x = 18.46)
Mastery learning vs. Macroadptation?
Student model
A pointer to next task
Attribute-value pairs
Student data, such as standardized test score, major, etc
Step analyzer and generator