1. Rika Yoshii, Ph.D. and Jacquelyn Hernandez ryoshii@csusm.edu CSIS Department California State University, San Marcos Send us suggestions and requests to use the system ItsLEADR: INTELLIGENT TUTORING SYSTEM FOR LEARNING ENGLISH ARTICLES BY DIAGRAMMATIC REASONING CALICO 2015 send comments to ryoshii@csusm.edu

2.  Our Goals  Background  Speaker’s Intent  Diagrams  Pedagogy  Domain Model  Student Model  Preliminary Evaluation  Summary  Future Work TOPICS send comments to ryoshii@csusm.edu

3.  Help ESL students develop reasoning skills in choosing English articles such as “a/an” and “the.”  Avoid the use terms such as “specific” and “definite” which are foreign concepts for students whose native languages do not contain an article system.  Must emphasize the speaker’s intent. “Speaker’s Intent determines the article” (Brown, 1973)(Celce-Murcia 1983) OUR GOALS send comments to ryoshii@csusm.edu

4. Conducted experiments with ESL students at CSUSM to set our goals. Started with the DaRT system (CALICO 1998): 1.Incorporated Diagrammatic Reasoning 2.Showed the intent of the speaker Students practiced understanding the reasons behind article usage by looking at diagrams. BACKGROUND send comments to ryoshii@csusm.edu

5. 1.We refined the speaker’s intent types. 2.We simplified the intent depicting diagrams, based on feedback from students at CSUSM. 3.We incorporated the pedagogical model, the domain model and the student model to provide individualized tutoring and to enforce mastery learning. 4.We continuously improved the graphical user interface based on feedback from test users. STEPS TO THE CURRENT INTELLIGENT SYSTEM send comments to ryoshii@csusm.edu

6.  7 intent types for “What is the speaker trying to communicate through the use of an article?” : 1.In Current Focus 2.Introduction 3.Asking Awareness 4.General Statement 5.Category is Important 6.Entailment for Whole 7.Entailment for Part SPEAKERS INTENT TYPES send comments to ryoshii@csusm.edu

7. In Current Focus– The speaker is communicating that the referent is in current focus on the discussion between the speaker and the listener. E.g. “And the teacher walked out.” Entailment for Whole - The speaker is communicating that he is referring to a complete part of another object. E.g. “Please change the tires.” SPEAKER’S INTENT – EXPLAINED (SINGULAR “THE”, PLURAL “THE”) send comments to ryoshii@csusm.edu

8. Introduction– the referent is being introduced by the speaker to the listener. E.g. “I saw a cute dog today.” Asking Awareness– the speaker is asking if the listener has the knowledge of the referent. E.g. “Did you see a man running that way?” General Statement – the speaker is making a general statement about a category. E.g. “Tigers are dangerous.” Category is Important – the speaker wants to emphasize the category more than a particular instance of it. E.g. “Hand me a pen.” Entailment for Part – the speaker is referring to an incomplete part of another object. E.g. “Please change a tire.” SPEAKER’S INTENT – EXPLAINED (SINGULAR “A”, PLURAL NONE) send comments to ryoshii@csusm.edu

9.  Is the referent a category, a group, or its member(s)?  Is the referent currently known by the speaker alone or by both the speaker and the listener?  Is the referent entailed by the referent of another noun phrase? SPEAKER’S INTENT - COMPONENTS send comments to ryoshii@csusm.edu

10. Intent components are depicted by:  The shape of the noun node  The image of the link node  The location of the noun node in the Venn Diagram DIAGRAMS send comments to ryoshii@csusm.edu

11.  A concrete element  A concrete group  A conceptual category NOUN NODES send comments to ryoshii@csusm.edu

12. LINKS IS-link is used when a noun phrase refers to the noun node itself. IS-IN-link is used when a noun phrase refers to some member(s) of the noun node. IS-ALL-OF-link is used when a noun phrase refers to all members of a noun node. IS-REP-link is used when a noun phrase refers to a representative of a noun node. send comments to ryoshii@csusm.edu

13. GENERAL STATEMENT DIAGRAM IS-LINK - CATEGORY - SHARED send comments to ryoshii@csusm.edu

14. GENERAL STATEMENT DIAGRAM IS-REP-LINK - CATEGORY - SHARED send comments to ryoshii@csusm.edu

15. IN CURRENT FOCUS DIAGRAM IS-LINK – ELEMENT - SHARED send comments to ryoshii@csusm.edu

16. INTRODUCTION DIAGRAM IS-LINK –ELEMENT - SHARED send comments to ryoshii@csusm.edu

17. CATEGORY IMPORTANT DIAGRAM IS-IN-LINK –CATEGORY - SHARED send comments to ryoshii@csusm.edu

18. ENTAILMENT DIAGRAM IS-ALL-OF – ENTAILED GROUP - SHARED send comments to ryoshii@csusm.edu

19.  3 phases:  Introduction – introduces intent types, diagrams and their components and asks review questions to make sure the student understands them.  Diagram selection - an intent type and a sentence are given, and the student is asked to choose the correct diagram.  Article selection - an intent type and a diagram are given, and the student is asked to fill in a blank space in the exercise sentence with an article. If the Help button is clicked, then a pop up window reviewing the diagrams and their components will appear. PEDAGOGY send comments to ryoshii@csusm.edu

20.  The student cannot move on to the next intent type until the mastery for that type is demonstrated by repeatedly answering correctly.  The student cannot move onto the next phase until the mastery of that phase is demonstrated.. MATERY LEARNING send comments to ryoshii@csusm.edu

21. 1.Domain Model 2.Student Model INTELLIGENT TUTOR COMPONENTS send comments to ryoshii@csusm.edu

22.  Expert knowledge of the subject  Used to check student answer against  Used to generate exercises  Represented as a semantic net that contains the intent types and corresponding diagram components.  DOMAIN MODEL send comments to ryoshii@csusm.edu

23. PART OF DOMAIN MODEL send comments to ryoshii@csusm.edu

24. Used to achieve individualized tutoring. Components: Student performance record – perturbed version of the domain model. Predictability model – probabilistic inference for predicting the student answers based on performance. Hint table - composed of the question type, the correct answer, the expected student answer, and corresponding hints addressing the student’s misconception. STUDENT MODELING send comments to ryoshii@csusm.edu

25.  If the expected student answer is an incorrect answer, the system will retrieve a hint from the hint table to present along with the question to the student.  If the student answers incorrectly, the system will retrieve a hint from the hint table to help the student.  Performance record is always updated.  Once the student has successfully mastered a phase, the system computes a student performance summary. HOW THEY ARE USED send comments to ryoshii@csusm.edu

26. EXAMPLE PERFORMANCE SUMMARY send comments to ryoshii@csusm.edu

27.  To help us identify areas of improvement in terms of the diagrams as well as the system features.  Nine ESL/EFL students at CSUSM (Chinese, Japanese and Korean) 1.Baseline survey 2.Use ItsLEADR for one hour 3.Feedback survey 4.Interview PRELIMINARY EVALUATION send comments to ryoshii@csusm.edu

28.  Average of 3.44 (5 being the best) when rating their belief that the ItsLEADR system is useful.  Commented that the diagrams help to give context where as memorization was the main mechanism in classrooms.  Found the links to be difficult to understand and remember.  Most participants did not read the introductory information and relied on the Help button. EVALUATION RESULTS send comments to ryoshii@csusm.edu

29.  Revise the links and/or link definitions and images to make them easy to understand and remember.  Create a training manual written in English.  Add online help features such as balloons and visual walkthrough. SUGGESTED BY EVALUATORS send comments to ryoshii@csusm.edu

30.  Enhanced DaRT from a CALL system to an intelligent tutoring system.  Refined the intent types.  Improved the diagrams.  Individualized tutoring by predicting student answers.  Developed in C++ with Qt for GUI.  Performed preliminary evaluation. SUMMARY send comments to ryoshii@csusm.edu

31.  Enhance the prediction of student answers by determining the root cause of errors and use this information in giving hints and selecting remedial exercises.  A larger scale formative evaluation followed by further improvements.  A summative evaluation with experimental groups and a control group, comparing improvements and retention over several months.  Please send suggestions and requests to use the system to ryoshii@csusm.edu FUTURE WORK send comments to ryoshii@csusm.edu