1. ICat as a Companion Robot Siska Fitrianie Dragos Datcu Alin Chitu L.J.M Rothkrantz {s.fitrianie, d.datcu, a.g.chitu, l.j.m.rothkrantz}@ewi.tudelft.nl http://mmi.tudelft.nl

2. Man-Machine-Interaction http://mmi.tudelft.nl 2 Human AIBO Interaction http://mmi.tudelft.nl/~aibo

3. Man-Machine-Interaction http://mmi.tudelft.nl 3 Requirements and Challenges Requirements: –Develop a system architecture for a companion cat –Design and develop multimodal system to portray emotions in iCat as a result of interaction with humans. –Design of a personality model adapted to iCat. Challenges: –Development of a complete set of an HCI system: input sensing, input processor, input fusion, dialog management, output fission. –Use of iCat’s specific software that is new and still under development –Little is still known or standardized in the cognitive model for humans.

4. Man-Machine-Interaction http://mmi.tudelft.nl 4 Requirements in other words … 9:00 AM 11:00 AM14:00 AM16:00 AM iCat! What’s today whether? iCat!, Let’s play tic tac toe! iCat.. Read me a story, please.. 8:00 AM Get a life! I am still sleeping! What was that?? Hey.. where is everybody?? I feel so lonely!!! I wish I have a companion … Are you out of your mind? I am sleeping!!! Today is Monday, 27 March 2006.. It will be bright sunshine but a bit cold … Hurray!! Finally I have a friend!! OK!! I do the first move! OK, ok! I’ll do it. Don’t cry..

5. Man-Machine-Interaction http://mmi.tudelft.nl 5 Multimodal Human Computer Interaction

6. Man-Machine-Interaction http://mmi.tudelft.nl 6 Fusion Input (Feature Level) Audio-Video Fusion for Automatic Speech Recognition

7. Man-Machine-Interaction http://mmi.tudelft.nl 7 Emotion Analysis: Emotion Extraction from Text (1)

8. Man-Machine-Interaction http://mmi.tudelft.nl 8 Emotion Analysis: Emotion Extraction from Text (2) 2D space of Affective Lexicon database based on relative distance of words meaning from WordNet and Multidimensional Scaling

9. Man-Machine-Interaction http://mmi.tudelft.nl 9 Emotion Analysis: Emotion Extraction from Facial Information

10. Man-Machine-Interaction http://mmi.tudelft.nl 10 Emotion Analysis: Emotion Extraction from Speech Prosody

11. Man-Machine-Interaction http://mmi.tudelft.nl 11 Dialog Action Manager

12. Man-Machine-Interaction http://mmi.tudelft.nl 12 Fission Output

13. Man-Machine-Interaction http://mmi.tudelft.nl 13 Fission Output: Language Generator Template-based (includes dialog act and emotion as parameters) TOPIC-HISTORY INPUT...... //other sentences... //other categories...//other categories with less available info... // other categories with different appraisal type... //other categories for different user-group... //different topic... // different dialog-act

14. Man-Machine-Interaction http://mmi.tudelft.nl 14 Fission Output: Emotion Generator (nPME model) - 1

15. Man-Machine-Interaction http://mmi.tudelft.nl 15 Fission Output: Emotion Generator (nPME model) - 2 Maslow Pyramid of Needs OCEAN Model of Personality: Openness, Conscientiousness, Extraversion, Agreeableness, Neuroticism Mood: Level of valence and arousal. Goals, Preferences, Standards (Rule-based) 41 Emotion Types and 3 Activation Levels

16. Man-Machine-Interaction http://mmi.tudelft.nl 16 The Point Symmetrical I/O modalities of user-system interaction Modular and general, as a starting point of any CHIM Possible applications: supervising, entertainment, tutor or help desk robot Current project development milestone: Input Sensing and Emotion Analysis