Quincy BrownKallen Tsikalas Research Questions & Hypotheses Theoretical Assumptions: Good, Bad & Ugly Using CTAT to test hypotheses The Interface Beneath the Interface: Models & Behavior Graphs Lessons Learned Extensions to the CTAT Interface Tools Future work An Experiment Using CTAT to Explore the Role of Self-Regulation in the Robust Learning of Middle School Math

Research Questions & Hypotheses 1.Effect of providing a self-regulatory goal. What is the effect of giving students an explicit self- regulatory goal [to be “error detectives”] on their robust learning and the accuracy of their self-efficacy ratings? 2.Effect of providing self-regulatory feedback and practice opportunities. What is the effect of providing students with feedback on and practice with a self-regulatory skill [error detection and correction] on their robust learning and the accuracy of their self- efficacy ratings? 3.Predictive power of accurate self-efficacy ratings. To what extent does the accuracy of students’ self-efficacy ratings effect their learning curve and help-seeking behavior? Outcome Variables - Accuracy of self-efficacy ratings - Learning curves from CTAT data - Pre-, post-, and delayed post-test scores How sure are you that you can solve this problem? Likert scale (1-10)

Theoretical Assumptions  Interventions that target students’ self- regulatory processes can lead to improved cycles of learning and improved academic and non-academic outcomes.  Examples of self-regulatory interventions are training and/or feedback on motivational beliefs, goal-setting, monitoring, self-judgments, etc.  Providing feedback on self-regulatory skills effects students’  Ability to create internal feedback and self-assess  Attributions about success or failure  Proficiency at help-seeking  Willingness to invest effort in dealing with feedback information  Cognitive load theory may suggest that attending to errors introduces extraneous load which may diminish robust learning.

Using CTAT to Test Hypotheses  2x2 factorial design  Control condition = Cognitive Tutor with no self-regulation enhancements’  Opportunities for assisted practice of cognitive skills  Multiple versions of Cognitive Tutor Self-Regulatory Goal +- -Control: CogTutor w/ no SR enhancements Error ID Feedback

The Interface Two Versions  Example-Tracing Tutor  Executed in Flash  Steps on separate screens  Dynamic feedback: Students have opportunity to interact with feedback screens  Full Cognitive Tutor  Executive in Flash  Interface represents deep mathematical structure

The CTAT Example-Tracing Interface  Executed in Flash  Steps on separate screens (Flash frames)  Dynamic feedback: Students have opportunity to interact with error feedback on screens (through Flash movies)

The CTAT Cognitive Tutor Interface  Executed in Flash  Streamlined format representing deep structure of mathematics

The CTAT Full Cognitive Tutor Behavior Graph Conflict Tree Working Memory Cognitive Model

The CTAT Full Cognitive Tutor Production Rules  All production rules functioning

The CTAT Example-Tracing Behavior Graph for the CogTutor Interface

Lessons Learned  How to use the CTAT tools  Importance of think-alouds for building example-tracing and production rules ­To create correct branching structure ­To optimize the number of rules – not more than needed  Potential threats to the efficacy of our intervention: Ken’s talk on design principles  Ideas about new types of learning outcomes (learning curves, help requests that lead to greater learning)

Extensions to CTAT Interface Tools  Multiple screens for one tutor  Navigation between screens that communicates with CTAT ­Via ActionScript ­Intratutor communication  Separate functions (e.g., visible and invisible Flash movies) for displaying feedback  Adjustments to Flash Widgets  Widgets just to log student actions/ideas rather than to tutor  Debugging of Flash tutorials

Future Work  Extension to mobile devices  Use of student characteristics (e.g., self- efficacy ratings) to guide specific tutoring actions  Use of student characteristics (e.g., accuracy of self-efficacy ratings) to predict learning curves

Special Thanks to… Everyone who helped us figure out what’s going on!  John and Brett for assistance with Flash widgets and communication between Example- Tracing functions and Flash interface  Jonathan and Vincent for assistance with full cognitive tutor development and production  Noboru for assistance with SimStudent  The PLSC Summer School students and staff for their good humor and great ideas!