1. Equal-party Conversation System for Language Learning Chih-yu Chao (advisor: Stephanie Seneff) April 14 th, 2006 Dialogs on Dialogs Reading Group

2. April 14th, 20062 Overview Motivation Approach Implementation Future Work References

3. April 14th, 20063 Motivation (1 of 3) Computer-Assisted Language Learning –Reading & writing –Listening (few) –Speaking (fewer) –Conversation (none?!)

4. April 14th, 20064 Motivation (2 of 3) Human tutors vs. computers –Confidence, stress –Making mistakes –Patience –Accessibility –Feedback

5. April 14th, 20065 Motivation (3 of 3) The proposed system –Domain-specific conversations –Sentence formation –Speaking –Listening –Less stressful environment –Game-like interaction process Remember the new vocabulary/sentence patterns more efficiently

6. April 14th, 20066 Approach (1 of 5) Domain –Activity scheduling (cf. database-accessing) Part of the lesson plan –Not simply form-filling Activity proposal Confirmation of likes/dislikes Activity negotiation Schedule checking Multiple turns of these actions

7. April 14th, 20067 Approach (2 of 5) Taking the initiative –Forming different types of sentences by playing a different role in each dialogue

8. April 14th, 20068 Approach (3 of 5) Languages of interest –Mandarin Chinese  English The game –Persona A schedule from yesterday to day after tomorrow Preference of activities –The virtual buddy also has a persona represented internally –Respond and/or negotiate based on the given persona until an agreement is reached –New persona

9. April 14th, 20069 :setup ( {c like :min_count 3 :max_count 5 :choices ( "swimming" "basketball" "television" "movies" "football" "baseball" "tennis" "dancing" "video games" "shopping" ) :unique 1 :pattern ":choices" } {c dislike :min_count 3 :max_count 5 :choices ( "swimming" "basketball" "television" "movies" "football" "baseball" "tennis" "dancing" "video games" "shopping" ) :unique 1 :rule_out "like" :pattern ":choices" } {c schedule :min_count 3 :max_count 7 :choices ( "swimming" "basketball" "television" "movies" "football" "baseball" "tennis" "dancing" "shopping" "studying" "video games" ) :ordinates ( ":day" ":time" ) :pattern ":choices" :day ( "yesterday" "today" "tomorrow" "day after tomorrow") :time ( "morning" "afternoon" "evening" ) :rule_out "dislike" } ) :features {c features :day ( "yesterday" "today" "tomorrow" "day after tomorrow") :time ( {c morning :start_hour 9 :end_hour 12 } {c afternoon :start_hour 13 :end_hour 17 } {c evening :start_hour 19 :end_hour 22 } )

10. April 14th, 200610 A: "what are you doing day after tomorrow afternoon?" B: "I am going to study the day after tomorrow in the afternoon" A: "are you free day after tomorrow morning?" B: "yes I am going to do nothing the day after tomorrow in the morning" A: "would you like to play tennis with me the day after tomorrow in the morning" B: "yes I do like to play tennis" A: "okay let's get together the day after tomorrow in the morning to play tennis" B: "yes that would be great" ====================== New Persona ====================== A: "what are you doing tomorrow evening?" B: "I am going to do nothing tomorrow evening" A: "would you like to play football with me tomorrow evening" B: “to play football is not bad" A: "okay let's get together tomorrow evening to play football" B: “okay to play football is not bad"

11. April 14th, 200611 Approach (5 of 5) Other features –Get help by speaking in the native language 2 recognizers, threshold –Score –Keeps track of the difficulties Same material (vocabulary or sentence pattern) will be reviewed until the user succeeds

12. April 14th, 200612 Implementation (1 of 3) Interlingua-based approach –Natural language input  language- independent semantic frame –TINA: syntactic and semantic analyses –Genesis: semantic frames  well-formed surface strings in the target language Based on the language-dependent generation rules (Pluto)

13. April 14th, 200613 {c verify :auxil “link” :topic {q pronoun :name “you” :number “pl” :domain “LanguageLesson” :sro “are you free day after tomorrow afternoon” :ilang “english” :olang “pinyin” :num_oreds 2 :pred {p attribute :topic “free” } :pred {p temporal :topic {q rel_date :name “day after tomorrow” } :num_preds 1 :pred {p daypart :topic {q daytime :name “afternoon” } } } }

14. April 14th, 200614 Implementation (3 of 3) 2 simulated users –Dialogues automatically generated by the system –Without eliciting input from human subjects –Acquire a large amount of data with very little cost –Good for different types of bootstrapping Coverage of language generation rules Accuracy of speech recognition Quality of speech synthesis

15. April 14th, 200615 Future Work Interface –Phone & web –Web only (with audio GUI) Peer-to-peer interaction mode –(Delayed) interaction between 2 learners –Target language only User study Curriculum design Evaluation –Software –Pedagogy

16. April 14th, 200616 References [1] V. Zue, S. Seneff, J. R. Glass, J. Polifroni, C. Pao, T. J. Hazen, and I. L. Hetherington. Jupiter: A telephone-based conversational interface for weather information. IEEE Trans. on Speech and Audio Processing, 8(1):100--112, 2000. [2] S. Seneff and J. Polifroni. Dialogue Management in the Mercury Flight Reservation System. In Proceedings of ANLP-NAACL 2000, Satellite Workshop, pp. 1--6, Seattle, WA. [3] S. Seneff. TINA: A Natural Language System for Spoken Language APplications. In Computational Linguistics, 18(1):61--86, 1992. [4] L. Baptist and S. Seneff. GENESIS-II: A Versatile System for Language Generation in Conversational System Applications. In Proceedings of ICSLP, pp. 271--274, Beijing, China, October 2000. [5] B. Cowan. PLUTO: A Preprocessor For Multilingual Spoken Language Generation. Master's thesis, MIT, Cambridge, MA, February 2004.

17. Questions?