1. SpeechLogic & NISLab ADS’04 2004-06-15 Design and First Tests of a Chatter Hans Dybkjær SpeechLogic™, Prolog Development Center A/S & Laila Dybkjær NISLab, University of Southern Denmark

2. SpeechLogic & NISLab ADS’04 2004-06-15 Chatting Dialogue type not common in state-of-the-art –Eliza, chatbots: written interaction New kinds of application –edutainment –chat with character from commercials series –small-talk while waiting instead of music Test-bed for new conversational techniques –express feelings –understand feelings –non-task oriented dialogue –other new features How far can we push current technology towards free conversation?

3. SpeechLogic & NISLab ADS’04 2004-06-15 Kurt Entertain users through chat (in Danish) Limited vocabulary (350 words) Phone-based Preferences of food, notably fruit and vegetables Kurt, e.g. his name, his age, and where he works Personality –childish –affective –self-centred –defensive with an underlying uncertainty –evasive Personality designed to hide shortcomings of understanding level

4. SpeechLogic & NISLab ADS’04 2004-06-15 Features for emotion modelling Available (Phonetic) lexicon Grammar Recognition scores Phrasing Dialogue flow Available, but not used: –n-best ambiguity –barge-in event handling –complex task domain Not available (input) Glottal stop Stress Prosody Non-linguistic vocal phenomena, e.g. laughter Mood (anger, joy,...) Aware sites Overlapping speech (back-channelling)... Platform allows limited emotion modelling features

5. SpeechLogic & NISLab ADS’04 2004-06-15 Interaction model Standard dialogue model extended with affective state and handling Manage dialogue Compute affect Generate output You are stupidFool yourself, … Linguistic personality Flow model s t a t e

6. SpeechLogic & NISLab ADS’04 2004-06-15 affect Linguistic personality Context-independent assumption manage output stupidFool, … flow s t a t e pers’lity Lexicon tagged with Face value Preference Embarrassment Used for Input interpretation Face value Kurt sensitive to losing face Negative face value: e.g. corrections and insults Positive face value: e.g. praise Preference Words are liked, disliked or neutral Embarrassment Certain words embarrassing All other words neutral

7. SpeechLogic & NISLab ADS’04 2004-06-15 Negation Changes face value and preference Does not affect embarrassment Syntactic negation: –you are not stupid Semantic negation: –you hate apples Implication of negation may depend on question or statement –you hate apples = don’t you hate apples –you are not stupid ≠ aren’t you stupid Though = and ≠ are not fully semantically correct, they hold with respect to face value and preference More complex logic negation not useful for spoken language

8. SpeechLogic & NISLab ADS’04 2004-06-15 pers’lity Affect computation Simplified but transparent manage output stupidFool, … flow s t a t e Self-confidence Recognition scores Changed by accept/reject Embarrassment Means topic change Face value Complex, simplify: –if any negative input, take minimum –otherwise take maximum Preference Positive/negative face value => knock-on effect Not a function of single words But: –if any negative input, take minimum –otherwise take maximum affect

9. SpeechLogic & NISLab ADS’04 2004-06-15 affect pers’lity Affective state Two-parameter model manage output stupidFool, … flow Self-confidence Influences –magnitude of satisfaction changes –flow Satisfaction Main personality control –scale from angry (low) to exalted (high) Overflow at both ends Initial level is neutral Changes computed from –input preference –input face value –self-confidence level s t a t e AngryExaltedCurrent HangupGet Angry

10. SpeechLogic & NISLab ADS’04 2004-06-15 s t a t e affect pers’lity Dialogue management Simple task solving plus some more chat-like interaction output stupidFool, … Flow model Questions Answers Statements Jokes Feedback –implicit, explicit Embarrassment Joke and change topic Satisfaction ”Underflow” leads to hangup No other flow effect Self-confidence flow manage 0lowhigh1 ImplicitExplicitFeedback:None At accept:Joke None medium

11. SpeechLogic & NISLab ADS’04 2004-06-15 flow manage s t a t e affect pers’lity Generate output A simple scheme with large variability stupidFool, … Phrases Canned Composed of: –Change marker –Insults and jokes –Answers and feedback –Prompts Change marker Notifies user of system’s emotional state Function of satisfaction state and satisfaction change –High, high: Happy –Low, low: Angry –High, low: Forbearing –Low, High: Distrustful Random phrases Variation, less rigid output

12. SpeechLogic & NISLab ADS’04 2004-06-15 Example dialogue No.UtteranceChange le.Sat.Act type U4You are stupid0 S5Did you say ”You are blue”? 0Explicit feedback U5No, you are stupid S6.1HnChange mark- er (cross) S6.2You did say ”You are stupid” didn’t you? Explicit feedback U6Yes S7.1If only you’d find half a snail in your salad Change mark- er (cross) S7.2NoAnswer Holluja, what a fool you are Prompt (passive)

13. SpeechLogic & NISLab ADS’04 2004-06-15 Data collection No controlled experiments Dialogues collected from demo-line 86 dialogues transcribed from 3 system iterations Many dialogues performed by children First output voice by 40 years old male Second output voice by 14 years old boy Small but sufficient to give impression

14. SpeechLogic & NISLab ADS’04 2004-06-15 Learned from dialogues (1) Start –identity –age –location –knows about –how are you During call –mostly questions concerning Kurt –maybe search for common ground –little volunteered information –dialogue on the conversation Dinner party conversation with a twist

15. SpeechLogic & NISLab ADS’04 2004-06-15 Learned from dialogues (2) Topics asked about by users –personal (where he works, where he lives, childhood, wife, children, health, hair, eye-colour, glasses, smokes, …) (parents, …) –adjective descriptions (stupid, clever, handsome, …) –likes and dislikes (alcohol, food, football, music, work, sex, …) –utterances related to what the system says (insults, long input, …) Topics depends on modelled person

16. SpeechLogic & NISLab ADS’04 2004-06-15 Next steps Extend grammar coverage Extend Kurt’s knowledge about himself Provide him with interests Let Kurt ask questions about the user Experiment with addition of new parameters (patience, balance, self-esteem, pessimism/optimism) Weighting of parameters depends on personality New kinds of interaction patterns (hand over phone, detection of repeated calls from same number) Extended conversational and emotional coverage

17. SpeechLogic & NISLab ADS’04 2004-06-15 Conclusion Clearly too small vocabulary and grammar for longer interactions Entertaining despite all shortcomings In particular –repetition of what was understood –reactions to insults Simple but entertaining aspects