2. Affective Computing and Human Robot Interaction a short introduction a short introduction Joost Broekens Telematica Institute, Enschede, LIACS, Leiden University.

3. Overview What is emotion? What theoretical angles can we take to study emotion? What is affective computing + examples ? Why is affective computing useful? 15 min coffee break. Interactive Human-Robot Interaction Human emotional expressions as feedback to a learning robot

4. Emotion (what is it)? Common emotions: fear, anger, happiness, sadness, surprise, disgust Short episode of synchronized system activity triggered by event: –subjective feelings (the emotion we normally refer to), –tendency to do something (action preparation), –facial expressions, –evaluation of the situation (cognitive evaluation, thinking), –physiological arousal (heartbeat, alertness). Affect (affectie)= related to emotion and mood: –emotion: short term, high intensity, object directed, –mood: unfocused, long term, low intensity, –affect: sometimes seen as abstraction of emotion/mood in terms of, positiveness/negativeness and activation/deactivation (Russell).

5. Emotion (why do we have it)? Situational evaluation and communication. Heuristic relating events to personal goals, needs and beliefs: –evaluates personal relevance of event (Scherer) and helps decision- making (Damasio), –fast reactions and action preparation (Frijda), –influence information processing and decision making (Damasio): e.g, adaptation, attention, search. Communication medium: –communicate internal state (show feelings), –alert others (direct attention), –show empathy (show understanding of situation). –Give feedback (evaluate behavior).

6. Emotion (what does it do)? Emotion and affect influence thought and behavior: –The kind of thoughts we have Mood congruency (e.g., positive moods triggers positive thought) –The way we process information Broad vs. narrow look (Goschke & Dreisbach) (forest vs. trees) A lot vs. a little processing effort (Scherer, Forgas) –What we think about things Emotion/mood as information (Clore & Gasper) (how does this feel?) Emotion as belief anchor (Frijda & Mesquita) (idea “fixation”) –How we learn and adapt Emotion/affect as social reinforcement (emotion expression as signal) Emotion/affect as intrinsic reinforcement (use my own emotion as feedback) Emotion as “metaparameter” to control learning process (explore vs. exploit)

7. Emotion (how can we study it)? Biological/Neurological (Damasio, Panksepp, LeDoux, Rolls): –aim: find the necessary / sufficient brain areas and circuits involved in emotions /feelings / self-regulation / adaptation. Biological/Evolutionary (Darwin, Ekman): –aim: why emotions exist in the first place, what’s the utility of emotion. Cognitive/Psychological (Scherer, Frijda): –aim: understand relation cognition/emotion, why does event e results in emotion x while: the same event e may result in a different emotion, and other events may result in the same emotion x. Social (Ekman, and others): –aim: understand the role of emotion in communication.

8. What is affective computing? Computing that relates to, arises from, or deliberately influences emotions (Picard). Different types of computer models: –recognize emotions (perception, in), –interpret/elicit emotions (cognition/behavior, processing), –emotional influence on behavior (adaptation, attention), –show emotions (expression, out), and –complex mixtures of these… An affective computing system is –a system of computational processes that perceives, expresses, interprets, or uses emotions, –e.g. a robot, a virtual character, a tutor agent, a fridge, etc…

9. Some examples…

10. SIMS 2 (Electronic Arts) Entertainment: emotions are used to provide entertainment value.

11. Kismet (Breazeal) Social: Kismet, A framework, using a humanoid head expressing emotions, to study: –effect of emotions on human-machine interaction. –learning of social robot behaviors during human-robot play. –joint attention.

12. Mission Rehearsal Exercise (Gratch & Marsella) Cognitive: study the influence of artificial emotions on –planning mechanism of virtual characters, –training effect on trainees (emotion might enhance effect)

13. Assistive robotics Aibo (Sony, Japan) Entertainment robot I-Cat (Philips, NL) Robot assistant for elderly people Paro (Wada et al, Japan) Robot companion for elderly Huggable (MIT, USA) Robot companion for elderly

14. Why is affective comp. useful (1)? From a theoretical point of view… Advance emotion theory –simulated agents that elicit emotions to study possible psychological structure of emotions (Scherer), –emotion as artificial motivator (e.g. a bored robot that explores) (Canamero). Understand intelligence and adaptation –laws and rules are not sufficient for understanding or predicting human behavior and intelligence (e.g. how to sift thru many possible choices) (Damasio, Picard) –simulated learning agents influenced by emotions (e.g., emotion as reward) (Breazeal, Broekens), –Artificial Intelligence, Artificial Life?

15. Why is affective comp. useful (2)? From a practical point of view… Facilitate Human-Computer (Robot) interaction –use emotions in simulated-agent plans (to simulate human reasoning) (Gratch & Marsella), –communication and joint attention (Breazeal, MIT) –robot acceptation (Heerink, Telin) –Persuasive design (VR training, tutor agents (Gratch & Marsella, Nijholt)) –Interactive robot learning (second part of this lecture). Entertainment –Computer games such as the Sims (EA), Aibo Robot (Sony).

16. Affective Computing in short… Affect has different meanings related to emotion. –emotion: short term, high intensity, object directed, –mood: unfocused, long term, low intensity, –affect: sometimes seen as abstraction of emotion/mood in terms of, positiveness/negativeness and activation/deactivation. Affective computing is about computing with emotions –“a system of computational processes that perceives, expresses, interprets, or uses emotions” Why? –advance emotion theory, –understand intelligence, –facilitate Human-Computer (Robot) interaction, –entertainment.

17. Interactive Robot Learning (Real-time interaction to control robot behavior)

18. Interactive Robot Learning: a special case of HRI Goals Human Robot Interaction: –Develop more natural and effective ways of interacting between robots and humans. –Allow non-expert interaction. –Allow flexibility of robot behavior. How? –Gesture recognition –Cognitive robot architectures –Machine learning (artificial adaptation) –Affective computing Emotion recognition Emotion expression

19. Example, Guidance and Feedback Interactive robot learning –Learning by Example Imitation learning (see Breazeal & Scassellati) E.g., gesture repetition –Learning by Guidance (Thomaz & Breazeal) In general: future directed learning cues, such as Directing attention Communicating motivational intentions (e.g., anticipatory reward) Proposing actions –Learning by Feedback Learn by trial (+feedback) and error (-feedback) and exploration. Additional human-based reinforcement signal (Breazeal & Velasquez; Isbell et al; Mitsunaga et al; Papudesi & Hubert)

20. Why study Interactive Robot Learning? Robot perspective: Interactive robot learning –Facilitate human-robot interaction –Study learning and adaptation –Future: enable robots to interactively cooperate in an efficient manner with humans in ways that are natural to humans. Psychological perspective: Understanding mechanisms –How can affect influence learning? –How can affect influence attention processes? –Joint Human-Robot attention –Study learner-teacher relations –Etc..

21. EARL A framework to study the influence of Emotion on Adaptive behavior in the context of Reinforcement Learning: –emotion as reinforcement (reward/punishment), –emotion as influence on learning process, –emotion as information, –artificial emotion resulting from learning process

22. EARL A typical EARL setup: –Simulated robot in a –Maze learning (navigation) tasks –Reinforcement learning (RL) approach to robot learning Learn by trial (+feedback) and error (-feedback) and exploration. –Robot has own model of emotion –Robot head to express emotion –And… Webcam and emotion recognition to interpret emotions

23. Experiment: Human Affect as Reinforcement to Robot An experiment to study interactive robot learning Simulated Robot learns to find food in a maze. –Explore unknown environment (try actions). –Reinforcement Learning Feedback from environment based on taken actions + feedback=repeat - feedback=don’t repeat –Learn which sequence of actions gives best +feedback. Robot also uses human facial expressions as +/- feedback

24. Facial Expression as Reinforcement Affective signal as additional reinforcement –Web cam –Emotional expression analysis –Positive emotion (happy) = reward –Negative emotion (sad) = punishment –So: in addition to feedback from environment emotional expression is used in learning as r human, a social reward coming from the human observer Sad / happy - / + Robot behavior to screen

25. Reinforcement-based robot learning pathwall food human feedback 2 b Food (+) Agent Wall (-) Human Feedback Path Start

26. Experiment: results Test learning difference between standard agent and social agents –Standard agent uses rewards environment to update behavior. –Social agent uses rewards from environment and reward from human emotional signal. Results: –Social agent learns the path to the food quicker (just believe me, so that I don’t have to explain the graphs) Earl demo…

27. Interactive Robot Learning in short… A special case of Human Robot Interaction –Goal HRI: more efficient, flexible, personal, pleasant human robot interaction –Interactive Robot Learning: Imitation, Feedback, Guidance. Affective Interaction Feasible and useful –can be used as guidance (MIT robot studies), and –feedback (experiment shown). Why? –Robot perspective Facilitate human-robot interaction Study learning and adaptation –Psychological perspective How can affect influence learning? How can affect influence attention processes? Joint Human-Robot attention Study learner-teacher relations