Rika Yoshii, Ph.D. and Jacquelyn Hernandez CSIS Department California State University, San Marcos Send us suggestions and requests to use the system ItsLEADR: INTELLIGENT TUTORING SYSTEM FOR LEARNING ENGLISH ARTICLES BY DIAGRAMMATIC REASONING CALICO 2015 send comments to

 Our Goals  Background  Speaker’s Intent  Diagrams  Pedagogy  Domain Model  Student Model  Preliminary Evaluation  Summary  Future Work TOPICS send comments to

 Help ESL students develop reasoning skills in choosing English articles such as “a/an” and “the.”  Avoid the use terms such as “specific” and “definite” which are foreign concepts for students whose native languages do not contain an article system.  Must emphasize the speaker’s intent. “Speaker’s Intent determines the article” (Brown, 1973)(Celce-Murcia 1983) OUR GOALS send comments to

Conducted experiments with ESL students at CSUSM to set our goals. Started with the DaRT system (CALICO 1998): 1.Incorporated Diagrammatic Reasoning 2.Showed the intent of the speaker Students practiced understanding the reasons behind article usage by looking at diagrams. BACKGROUND send comments to

1.We refined the speaker’s intent types. 2.We simplified the intent depicting diagrams, based on feedback from students at CSUSM. 3.We incorporated the pedagogical model, the domain model and the student model to provide individualized tutoring and to enforce mastery learning. 4.We continuously improved the graphical user interface based on feedback from test users. STEPS TO THE CURRENT INTELLIGENT SYSTEM send comments to

 7 intent types for “What is the speaker trying to communicate through the use of an article?” : 1.In Current Focus 2.Introduction 3.Asking Awareness 4.General Statement 5.Category is Important 6.Entailment for Whole 7.Entailment for Part SPEAKERS INTENT TYPES send comments to

In Current Focus– The speaker is communicating that the referent is in current focus on the discussion between the speaker and the listener. E.g. “And the teacher walked out.” Entailment for Whole - The speaker is communicating that he is referring to a complete part of another object. E.g. “Please change the tires.” SPEAKER’S INTENT – EXPLAINED (SINGULAR “THE”, PLURAL “THE”) send comments to

Introduction– the referent is being introduced by the speaker to the listener. E.g. “I saw a cute dog today.” Asking Awareness– the speaker is asking if the listener has the knowledge of the referent. E.g. “Did you see a man running that way?” General Statement – the speaker is making a general statement about a category. E.g. “Tigers are dangerous.” Category is Important – the speaker wants to emphasize the category more than a particular instance of it. E.g. “Hand me a pen.” Entailment for Part – the speaker is referring to an incomplete part of another object. E.g. “Please change a tire.” SPEAKER’S INTENT – EXPLAINED (SINGULAR “A”, PLURAL NONE) send comments to

 Is the referent a category, a group, or its member(s)?  Is the referent currently known by the speaker alone or by both the speaker and the listener?  Is the referent entailed by the referent of another noun phrase? SPEAKER’S INTENT - COMPONENTS send comments to

Intent components are depicted by:  The shape of the noun node  The image of the link node  The location of the noun node in the Venn Diagram DIAGRAMS send comments to

 A concrete element  A concrete group  A conceptual category NOUN NODES send comments to

LINKS IS-link is used when a noun phrase refers to the noun node itself. IS-IN-link is used when a noun phrase refers to some member(s) of the noun node. IS-ALL-OF-link is used when a noun phrase refers to all members of a noun node. IS-REP-link is used when a noun phrase refers to a representative of a noun node. send comments to

GENERAL STATEMENT DIAGRAM IS-LINK - CATEGORY - SHARED send comments to

GENERAL STATEMENT DIAGRAM IS-REP-LINK - CATEGORY - SHARED send comments to

IN CURRENT FOCUS DIAGRAM IS-LINK – ELEMENT - SHARED send comments to

INTRODUCTION DIAGRAM IS-LINK –ELEMENT - SHARED send comments to

CATEGORY IMPORTANT DIAGRAM IS-IN-LINK –CATEGORY - SHARED send comments to

ENTAILMENT DIAGRAM IS-ALL-OF – ENTAILED GROUP - SHARED send comments to

 3 phases:  Introduction – introduces intent types, diagrams and their components and asks review questions to make sure the student understands them.  Diagram selection - an intent type and a sentence are given, and the student is asked to choose the correct diagram.  Article selection - an intent type and a diagram are given, and the student is asked to fill in a blank space in the exercise sentence with an article. If the Help button is clicked, then a pop up window reviewing the diagrams and their components will appear. PEDAGOGY send comments to

 The student cannot move on to the next intent type until the mastery for that type is demonstrated by repeatedly answering correctly.  The student cannot move onto the next phase until the mastery of that phase is demonstrated.. MATERY LEARNING send comments to

1.Domain Model 2.Student Model INTELLIGENT TUTOR COMPONENTS send comments to

 Expert knowledge of the subject  Used to check student answer against  Used to generate exercises  Represented as a semantic net that contains the intent types and corresponding diagram components.  DOMAIN MODEL send comments to

PART OF DOMAIN MODEL send comments to

Used to achieve individualized tutoring. Components: Student performance record – perturbed version of the domain model. Predictability model – probabilistic inference for predicting the student answers based on performance. Hint table - composed of the question type, the correct answer, the expected student answer, and corresponding hints addressing the student’s misconception. STUDENT MODELING send comments to

 If the expected student answer is an incorrect answer, the system will retrieve a hint from the hint table to present along with the question to the student.  If the student answers incorrectly, the system will retrieve a hint from the hint table to help the student.  Performance record is always updated.  Once the student has successfully mastered a phase, the system computes a student performance summary. HOW THEY ARE USED send comments to

EXAMPLE PERFORMANCE SUMMARY send comments to

 To help us identify areas of improvement in terms of the diagrams as well as the system features.  Nine ESL/EFL students at CSUSM (Chinese, Japanese and Korean) 1.Baseline survey 2.Use ItsLEADR for one hour 3.Feedback survey 4.Interview PRELIMINARY EVALUATION send comments to

 Average of 3.44 (5 being the best) when rating their belief that the ItsLEADR system is useful.  Commented that the diagrams help to give context where as memorization was the main mechanism in classrooms.  Found the links to be difficult to understand and remember.  Most participants did not read the introductory information and relied on the Help button. EVALUATION RESULTS send comments to

 Revise the links and/or link definitions and images to make them easy to understand and remember.  Create a training manual written in English.  Add online help features such as balloons and visual walkthrough. SUGGESTED BY EVALUATORS send comments to

 Enhanced DaRT from a CALL system to an intelligent tutoring system.  Refined the intent types.  Improved the diagrams.  Individualized tutoring by predicting student answers.  Developed in C++ with Qt for GUI.  Performed preliminary evaluation. SUMMARY send comments to

 Enhance the prediction of student answers by determining the root cause of errors and use this information in giving hints and selecting remedial exercises.  A larger scale formative evaluation followed by further improvements.  A summative evaluation with experimental groups and a control group, comparing improvements and retention over several months.  Please send suggestions and requests to use the system to FUTURE WORK send comments to