Case-Based Learning Mechanisms to Deliver Learning Materials Todd Blank, Leen-Kiat Soh, L. D. Miller, Suzette Person Department of Computer Science and Engineering University of Nebraska {tblank, lksoh, lmille,

The ILMDA Project: Goals Build an agent capable of adapting examples and problems to student behavior Build an agent capable of adapting examples and problems to student behavior Develop courseware for CS1 and CS2 computer science classes Develop courseware for CS1 and CS2 computer science classes Establish a flexible database for the ILMDA system Establish a flexible database for the ILMDA system Intelligent Learning Materials Delivery AgentIntelligent Learning Materials Delivery Agent

Learning Material Tutorial Tutorial RecursionRecursion Examples Examples Towers of HanoiTowers of Hanoi Problems Problems What is the output of this method if we pass in 5 as an argument?What is the output of this method if we pass in 5 as an argument?

Project Framework GUI front-end GUI front-end ILMDA reasoning module ILMDA reasoning module Database backend Database backend

ILMDA Reasoning student Computer & GUI database lectures Historical profile, Real-time behavior Parametric profile of student and environment Retrieval instructions Profile updates Statistics updates Timely delivery of examples & exercise problems Examples Exercise problems Statistics ILMDA Agent Project Framework

Design Features Utilization of true agent intelligence Utilization of true agent intelligence Agent accountability of usefulness for evaluation Agent accountability of usefulness for evaluation Modularization of course content and delivery Modularization of course content and delivery

Intelligence Modules Agent employs Agent employs Case-based ReasoningCase-based Reasoning Feedback from environmentFeedback from environment Meta-learns about good adaptation heuristicsMeta-learns about good adaptation heuristics

Learning Modules Case Learning Module Case Learning Module Similarity Learning Module Similarity Learning Module Adaptation Learning Module Adaptation Learning Module

Case Learning Module Finds most similar case Finds most similar case For successful cases, adapts on case with Case-based reasoningFor successful cases, adapts on case with Case-based reasoning For non-successful cases, adapts on case with Simulated AnnealingFor non-successful cases, adapts on case with Simulated Annealing Checks success rate of each case stored in database Checks success rate of each case stored in database timesCaseUsed, timesCaseSuccessfultimesCaseUsed, timesCaseSuccessful Unsuccessful cases have candidateForAnneal set to trueUnsuccessful cases have candidateForAnneal set to true

Similarity Learning Module Changes weights for computing similarity of two cases Changes weights for computing similarity of two cases Compares outcome for each case with its most similar caseCompares outcome for each case with its most similar case Weights are increased when cases have similar outcomesWeights are increased when cases have similar outcomes Weights are decreased when cases have dissimilar outcomesWeights are decreased when cases have dissimilar outcomes

Adaptation Learning Module Changes weights for adaptation on most similar case Changes weights for adaptation on most similar case Considers outcome of previously used casesConsiders outcome of previously used cases For successful cases, slight changes to adaptation weightsFor successful cases, slight changes to adaptation weights For non-successful cases, aggressive changes to adaptation weightsFor non-successful cases, aggressive changes to adaptation weights

Adaptation Learning Module Checks success rate cases stored in database Checks success rate cases stored in database Cases have been successful Cases have been successful Slight changes to adaptation weightsSlight changes to adaptation weights Cases have been unsuccessful Cases have been unsuccessful More aggressive changes to adaptation weightsMore aggressive changes to adaptation weights

Implementation ILMDA agent ILMDA agent Java (Swing for GUI)Java (Swing for GUI) PHPPHP MySQL databaseMySQL database Learning Modules Learning Modules JavaJava MySQL databaseMySQL database

Implementation MySQL database houses MySQL database houses Student InformationStudent Information TutorialsTutorials ExamplesExamples ProblemsProblems Agent HeuristicsAgent Heuristics Performance StatisticsPerformance Statistics

Student Interaction with GUI

A Case: Input Features Input VariablesDescription AveExmpClicksThe average number of times the student clicks the mouse in the examples he or she has seen. (When comparing two cases to pick a problem, the number of example clicks in that session is used instead). AveExmpTimeThe average time spent (in milliseconds) per example. (When comparing two cases to pick a problem, the amount of time spent during the example clicks in that session is used instead). AveExmpToTtrlThe average number of times the student goes back to the tutorial from the example. AveGradeThe student’s average grade on the problems. NumExmp, NumProb, TtrlClicks, TtrlTime, Self Efficacy, Motivation, etc

ParametersDescription DiffLevelThe difficulty level of the problem or example. MinUseTimeThe shortest anyone has looked at the problem or example. MaxUseTimeThe longest anyone has looked at the problem or example. AveUseTimeThe average time students view the problem or example. AveClickThe average # of times students click the mouse in the problem or example LengthThe # of characters in the example or problem BloomThe Bloom’s taxonomy value for the problem or example. A Case: Output Features

Simulator Virtual Student Types Virtual Student Types Student SpeedStudent Speed Slow Slow Average Average Fast Fast Student AptitudeStudent Aptitude Knowledgeable Knowledgeable Average Average Not Knowledgeable Not Knowledgeable

Experiments ILMDA interacts with Virtual student ILMDA interacts with Virtual student Measure average quitting point and outcome for Virtual students Measure average quitting point and outcome for Virtual students Quit at tutorial (0), Quit at example (1), Quit at problem (2), Quit after problem (3), Answer problem correctly (4)Quit at tutorial (0), Quit at example (1), Quit at problem (2), Quit after problem (3), Answer problem correctly (4) Measure correct answer scores Measure correct answer scores

Results 1000 simulations with and without learning modules 1000 simulations with and without learning modules Without learning modulesWithout learning modules average quitting point,.056 average score average quitting point,.056 average score With learning modulesWith learning modules average quitting point,.116 average score average quitting point,.116 average score Observation (Marginal) Observation (Marginal) ILMDA was able to give better examplesILMDA was able to give better examples ILMDA was able to give problems that virtual students answered correctly more oftenILMDA was able to give problems that virtual students answered correctly more often

Ongoing Work Deployed ILMDA in CSCE155, Fall 04 Deployed ILMDA in CSCE155, Fall 04 Five topics: Simple Class, Exception, Event- Driven Programming, Inheritance/Polymorphism, RecursionFive topics: Simple Class, Exception, Event- Driven Programming, Inheritance/Polymorphism, Recursion Agents with and without learning mechanismsAgents with and without learning mechanisms To find the impact of learning To find the impact of learning To identify features To identify features To identify key adaptation heuristics To identify key adaptation heuristics To identify useful cases To identify useful cases Results collected but yet to be analyzedResults collected but yet to be analyzed