Presentation is loading. Please wait.

Presentation is loading. Please wait.

Noboru Matsuda Human-Computer Interaction Institute

Published bySophie Kelley Modified over 8 years ago

Similar presentations

Presentation on theme: "Noboru Matsuda Human-Computer Interaction Institute"— Presentation transcript:

1 SimStudent: A Computational Model of Learning as a Research Toolbox for the Sciences of Learning
Noboru Matsuda Human-Computer Interaction Institute Carnegie Mellon University

2 Research Questions Building a cognitive model is hard. Can machine-learning techniques help non-expert authors build a Cognitive Tutor? Would like to simulate students’ learning. Can machine-learning techniques help us build a computational model of learning with a cognitive fidelity? I heard that students learn by teaching others. Can we use the computational model of learning to study the theory of learning by teaching?

3 Solution: SimStudent Machine learning agent Fundamental technology
Learns procedural skills, by Observing model solutions & solving problems Fundamental technology Programming by Demonstration Inductive Logic Programming Knowledge representation Production rules (Jess) Lau & Weld (1998). Blessing (1997).

4 SimStudent Projects Intelligent Authoring
Building a Cognitive Tutor as a CTAT Plug-in Student Modeling and Simulation Controlled educational studies Error formation study Prerequisite conceptual knowledge study Teachable Peer Learner Learning by teaching

5 Authoring a Cognitive Tutor
Example-Tracing Tutor Little programming A cognitive model specific to a particular problem Limited generalization by editing a behavior graph Model-Tracing Tutor Powerful student model Cognitive task analysis is hard Writing production rules is even more challenging Performing a task is relatively easy…

6 Next Generation Authoring
Build a tutor GUI Teaching a solution SimSt. learning Production Rules Rule simplify-LHS: IF is-equation( Eq ), is-lhs( Eq, Lhs ), polynomial( Lhs ), all-var-terms( Lhs ) Then simplify( Lhs, S-lhs ), enter( S-lhs ) Rule simplify-LHS: IF is-equation( Eq ), is-lhs( Eq, Lhs ), polynomial( Lhs ), all-var-terms( Lhs ) Then simplify( Lhs, S-lhs ), enter( S-lhs ) Rule simplify-LHS: IF is-equation( Eq ), is-lhs( Eq, Lhs ), polynomial( Lhs ), all-var-terms( Lhs ) Then simplify( Lhs, S-lhs ), enter( S-lhs )

7 Demo Authoring by Tutoring

8 Example: Learning to subtract a constant term
Learning to subtract a constant number First example Subtract the difference between 4 and 3… subtract 1 Subtract the coefficient of X… I see 3x, 1, x, and 4 in the equation. I wonder where the ‘1’ came from… Subtract the last term on the left-hand side… PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU)

9 Example: Learning to subtract a constant term
Learning to subtract a constant number First example Subtract the difference between 4 and 3… subtract 1 Subtract the coefficient of X… I see 3x, 1, x, and 4 in the equation. I wonder where the ‘1’ came from… Subtract the last term on the left-hand side… PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU)

10 PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU)
Prior Knowledge Feature predicates 18 predicates isFractionTerm(X), isConstant(X), isPolynomial(X),… Operators 42 operators add(X,Y), coefficient(X), getFrstNumber(X), … PSLC Summer School 2012 :: SimStudent :: Noboru Matsuda (CMU)

11 Example: Stoichiometry Tutor

12 SimStudent Projects Intelligent Authoring
Building a Cognitive Tutor as a CTAT Plug-in Student Modeling and Simulation Controlled educational studies Error formation study Prerequisite conceptual knowledge study Teachable Peer Learner Learning by teaching

13 Model of Incorrect Learning
Identify errors students commonly make Weaken SimStudent’s background knowledge Let SimStudent make an induction error

14 Weak Prior Knowledge Hypothesis
Multiple ways to make sense of examples Get a coefficient and divide Get a denominator and multiply “multiply by x” 3x=5 “divide by 3” 4/x=5 “divide by 4” Get a number and divide

15 Results: Learning Rate
Step Score Estimate of correct (probability of each step right) Precision -> step score # training problems Steps Score = 0 (if there is no rule applicable) # correct rule applications / # all rule applications

16 Student Model A set of knowledge components (KCs)
(Li et al. EDM2011) A Machine Learning Approach for Automatic Student Model Discovery A set of knowledge components (KCs) Encoded in intelligent tutors to model how students solve problems E.g. How to proceed given problems of the form Nv=N One of the key factors that affects automated tutoring systems in making instructional decisions Previous Approach: Require expert input Highly subjective Proposed Approach: Use a machine-learning agent, SimStudent, to acquire knowledge 1 Skill  1 KC Skill application for that step  KC for each step

17 Human-generated vs SimStudent KCs
Human-generated Model SimStudent Comment Total # of KCs 12 21 # of Basic Arithmetic Operation KCs 4 13 Split into finer grain sizes based on different problem forms # of Typein KCs Approximately the same # of Other Transformation Operation KCs (e.g. combine like terms) Arithmetic op are like “add to both sides”, subtract from both sides, etc. 4x = 20 vs. –x = 5

18 Results Significance Test
Human-generated Model SimStudent AIC 6529 6448 3-Fold Cross Validation RMSE 0.4034 0.3997 Significance Test SimStudent outperforms the human- generated model in 4260 out of 6494 steps p < 0.001 SimStudent outperforms the human-generated model across 20 runs of cross validation AFM Performance proportion = and odds Can you compute separately this ratio for the steps where the Q matrix is different from

19 SimStudent Projects Intelligent Authoring
Building a Cognitive Tutor as a CTAT Plug-in Student Modeling and Simulation Controlled educational studies Error formation study Prerequisite conceptual knowledge study Teachable Peer Learner Learning by teaching

20 Learning by Teaching SimStudent

21 Learn more about SimStudents
Project Web Contact us Noboru Matsuda

Download ppt "Noboru Matsuda Human-Computer Interaction Institute"

Similar presentations

Introduction to AuthorIT April 10, 2006 Symposium on Knowledge Representation TICL SIG Joseph M. Scandura, Ph.D. Chairman, Board Scientific Advisors, MERGE.

Introduction to AuthorIT April 10, 2006 Symposium on Knowledge Representation TICL SIG Joseph M. Scandura, Ph.D. Chairman, Board Scientific Advisors, MERGE.
Modelling with expert systems. Expert systems Modelling with expert systems Coaching modelling with expert systems Advantages and limitations of modelling.

Modelling with expert systems. Expert systems Modelling with expert systems Coaching modelling with expert systems Advantages and limitations of modelling.
Principal Component Analysis Based on L1-Norm Maximization Nojun Kwak IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008.

Principal Component Analysis Based on L1-Norm Maximization Nojun Kwak IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008.
What can CTAT do for you? Overview of the CTAT track Vincent Aleven, Bruce McLaren and the CTAT team 3rd Annual PSLC LearnLab Summer School Pittsburgh,

What can CTAT do for you? Overview of the CTAT track Vincent Aleven, Bruce McLaren and the CTAT team 3rd Annual PSLC LearnLab Summer School Pittsburgh,
Educational data mining overview & Introduction to Exploratory Data Analysis Ken Koedinger CMU Director of PSLC Professor of Human-Computer Interaction.

Educational data mining overview & Introduction to Exploratory Data Analysis Ken Koedinger CMU Director of PSLC Professor of Human-Computer Interaction.
Improving learning by improving the cognitive model: A data- driven approach Cen, H., Koedinger, K., Junker, B. Learning Factors Analysis - A General Method.

Improving learning by improving the cognitive model: A data- driven approach Cen, H., Koedinger, K., Junker, B. Learning Factors Analysis - A General Method.
An Individualized Web-Based Algebra Tutor D.Sklavakis & I. Refanidis 1 An Individualized Web-Based Algebra Tutor Based on Dynamic Deep Model Tracing Dimitrios.

An Individualized Web-Based Algebra Tutor D.Sklavakis & I. Refanidis 1 An Individualized Web-Based Algebra Tutor Based on Dynamic Deep Model Tracing Dimitrios.
Supporting (aspects of) self- directed learning with Cognitive Tutors Ken Koedinger CMU Director of Pittsburgh Science of Learning Center Human-Computer.

Supporting (aspects of) self- directed learning with Cognitive Tutors Ken Koedinger CMU Director of Pittsburgh Science of Learning Center Human-Computer.
Does Conjunctive Knowledge Tracing Provide Leverage to the Temporal and Location Heuristics in Error Attribution? Adaeze Nwaigwe University of Maryland.

Does Conjunctive Knowledge Tracing Provide Leverage to the Temporal and Location Heuristics in Error Attribution? Adaeze Nwaigwe University of Maryland.
Learning from Observations Copyright, 1996 © Dale Carnegie & Associates, Inc. Chapter 18 Spring 2004.

Learning from Observations Copyright, 1996 © Dale Carnegie & Associates, Inc. Chapter 18 Spring 2004.
A C OMPUTATIONAL M ODEL OF A CCELERATED F UTURE L EARNING THROUGH F EATURE R ECOGNITION Nan Li Computer Science Department Carnegie Mellon University Building.

A C OMPUTATIONAL M ODEL OF A CCELERATED F UTURE L EARNING THROUGH F EATURE R ECOGNITION Nan Li Computer Science Department Carnegie Mellon University Building.
Learning from Observations Copyright, 1996 © Dale Carnegie & Associates, Inc. Chapter 18 Fall 2005.

Learning from Observations Copyright, 1996 © Dale Carnegie & Associates, Inc. Chapter 18 Fall 2005.
LEARNING FROM OBSERVATIONS Yılmaz KILIÇASLAN. Definition Learning takes place as the agent observes its interactions with the world and its own decision-making.

LEARNING FROM OBSERVATIONS Yılmaz KILIÇASLAN. Definition Learning takes place as the agent observes its interactions with the world and its own decision-making.
Learning from Observations Copyright, 1996 © Dale Carnegie & Associates, Inc. Chapter 18 Fall 2004.

Learning from Observations Copyright, 1996 © Dale Carnegie & Associates, Inc. Chapter 18 Fall 2004.
SAT-Based Decision Procedures for Subsets of First-Order Logic

SAT-Based Decision Procedures for Subsets of First-Order Logic
6.2 Modelling the Learner ISE554 The WWW for eLearning.

6.2 Modelling the Learner ISE554 The WWW for eLearning.
Our Explorations with CTAT!

Our Explorations with CTAT!
LEARNING FROM OBSERVATIONS Yılmaz KILIÇASLAN. Definition Learning takes place as the agent observes its interactions with the world and its own decision-making.

LEARNING FROM OBSERVATIONS Yılmaz KILIÇASLAN. Definition Learning takes place as the agent observes its interactions with the world and its own decision-making.
The Implicit Mapping into Feature Space. In order to learn non-linear relations with a linear machine, we need to select a set of non- linear features.

The Implicit Mapping into Feature Space. In order to learn non-linear relations with a linear machine, we need to select a set of non- linear features.
+ Doing More with Less : Student Modeling and Performance Prediction with Reduced Content Models Yun Huang, University of Pittsburgh Yanbo Xu, Carnegie.

+ Doing More with Less : Student Modeling and Performance Prediction with Reduced Content Models Yun Huang, University of Pittsburgh Yanbo Xu, Carnegie.

Similar presentations

Ads by Google