Presentation is loading. Please wait.

Presentation is loading. Please wait.

Jonathan Gratch USC Institute for Creative Technologies Joint work with Stacy Marsella USC Information Sciences Institute Jonathan Gratch USC Institute.

Similar presentations


Presentation on theme: "Jonathan Gratch USC Institute for Creative Technologies Joint work with Stacy Marsella USC Information Sciences Institute Jonathan Gratch USC Institute."— Presentation transcript:

1 Jonathan Gratch USC Institute for Creative Technologies Joint work with Stacy Marsella USC Information Sciences Institute Jonathan Gratch USC Institute for Creative Technologies Joint work with Stacy Marsella USC Information Sciences Institute The Architectural Role of Emotions in Cognitive Systems

2 Outline Emotions are adaptive Can inform cognitive system design Ground in implemented cognitive system –Mission Rehearsal Exercise system –Cognitive Appraisal Theory –Illustrate impact on architecture design General implications for cognitive systems Outline

3 Revolutionary progress in emotion research –Neurophysiology of emotion (Damasio, LeDoux) –Appraisal theories (Frijda, Lazarus, Scherer) Emotions appear adaptive (in moderation?) –Decision-making ─ Focus of attention –Learning ─ Social relationships –Belief formation ─ Communication Growing interest in “emotional” systems with focus on modeling human behavior –HCI (non-verbal recognition and generation) –User and human behavior modeling –Believability/Entertainment Adaptive Role of Emotions

4 Outline Can inform intelligent behavior in general Motivate behavior Balancing competing goals Balancing reaction and deliberation Disambiguating stimuli in light of existing beliefs and commitments Abstract and formalize as information processing –Not new idea: Simon(1967), Oatley&Johson-Laird(1987), Sloman –Revisit in light of new findings In intelligent systems In theories of emotion Architectural Perspective

5 Grounding: Virtual Humans Face-to-face interaction Verbal & non-verbal behavior Swartout, Gratch, Hill, Hovy, Johnson, Marsella, Narayanan, Rickel, Traum, … Marsella, Johnson & Labore

6 Mission Rehearsal Exercise Social Training Simulation Explore high-stakes social interactions in safety of VR

7 Mission Rehearsal Exercise Team decision-making in crisis situations : –Non-scripted real-time interactions –Planning, replanning, and plan execution –Teamwork, distributed authority and responsibility –Collaborative, mixed initiative dialogue –Multi-party conversations –Verbal and non-verbal communication –Emotionally-biased behavior

8 Mission Rehearsal Exercise Assumptions/Limitations : –Tightly focused task-related dialogue –Near-expert decision makers –Stylized vocabulary (military speak) –Stylized virtual environment

9 MRE: Leadership Training

10 Speech Recognition (HTK) Semantic Parser Motion/ Gesture Scheduler (Beat) Text to Speech (Festival) World Simulator Animation System BDI Haptek Communication Bus Audio (Protools) Voice Input Vega Projection System Speakers (10.2) Soar Planning DialogueAction Selection Perception NLG Emotion NLU pragmatics

11 Child Healthy:False Accident Intend: False Blame: unresolved Assist Eagle 1-6:False Eagle 1-6 Assist Desire: LT Belief: False Child-Healthy Desire: SGT Belief: False Probability: 75% Get Medevac Responsibility:LT Intend: True Medevac Available:True Past FuturePresent Cognitive Representation Soar’s Working Memory PlanningPerceptionDialogueAction Soar operators

12 Child Healthy:False Accident Intend: False Blame: unresolved Assist Eagle 1-6:False Eagle 1-6 Assist Desire: LT(+50) Belief: False Child-Healthy Desire: SGT(+80) Belief: False Probability: 75% Get Medevac Responsibility:LT Intend: True Medevac Available:True Past Events Future PlansPresent Cognitive Representation Causal Interpretation –Combines decision-theoretic plans with models of belief and intention –Uniform representation of past, present, future –Agent centric subjective view

13 Architectural Role of Emotion Began with view “emotion as veneer” Ended up as central organizing construct –Initial problem: how to convey emotion in interactive setting? –Built mechanism to infer plausible emotions In response to simulation events In response to user interventions –But discovered resolved architectural issues Coherence is more than skin deep Build it and they will come

14 How to convey emotion Cognitive Appraisal Theory –Influential and well-established theory Arnold, Frijda; Lazarus; Ortony, Clore & Collins; Scherer; Smith –Emphasizes tight coupling between Emotion Cognition Motivation

15 Goals, Beliefs External Events Coping Emotion Appraisal Problem- focused Emotion- focused Cognitive Appraisal Theory Coping Smith and Lazarus’ cognitive-motivational-emotive system

16 Goals, Beliefs External Events Appraisal Appraisal = Situation assessment –Compare beliefs, desires and intentions with external circumstances

17 Appraisal Characterize via appraisal variables –Desirability –Likelihood –Urgency –Unexpectedness –Causal attribution (causality, agency, blame/credit) –Coping potential (controllability, adaptability) Superset of criteria considered by cog systems –Decision theory: desirability, likelihood –Scheduling: desirability, urgency

18 Coping Problem- focused Emotion- focused Coping = Response strategy –Characterized by ontology of coping strategies Emotion Coping Strategies Goals, Beliefs External Events

19 Problem-focused (act on the world) –Action execution –Planning –Seek instrumental social support Analogous to: –Deliberative or reactive problem solving –Team negotiation Coping Strategies

20 Emotion-focused (act on belief) –Denial –Find silver lining –Shift blame –Distancing Not typically considered by cog systems systems –More than a decision (e.g. abandon current plan) Provides self-justification for why Related to motivational / explanatory coherence Leads to persistent change in behavior Coping Strategies

21 Modeling Appraisal and Coping Past Future PlanningPerceptionDialogueAction Soar’s Working Memory Soar Operators

22 Modeling Appraisal and Coping Appraisal as plan-evaluation –Causal interpretation mediates agent-environment relationship –Define appraisal variables in terms of features of interpretation –Fast, reactive, parallel Coping as generalized plan critics Map to operators that change interpretation –Problem-focused  execute step, add plan step –Emotion-focused Denial  Change belief Find silver lining  Change utilities Shift blame  Change causal attribution  Dialogue moves Distancing  Drop goal / intention

23 Émile: Architectural Manifestation The Emotional Octopus

24 Child Healthy:False Accident Blame: unresolved Assist Eagle 1-6:False Eagle 1-6 Assist Desire: LT Satisfied: False Child-Healthy Desire: SGT Satisfied: False Distress: 80 Sgt’s Appraisal of Accident from his perspective Perspective: Self (Sgt) Desirability: -80 Likelihood: 100% Blame/Credit: unresolved Appraisal

25 Child Healthy:False Accident Blame: unresolved Assist Eagle 1-6:False Distress: 80 Child-Healthy Desire: SGT Satisfied: False Distress: 80 Perspective: Lieutenant Desirability: -80 Certainty: 100% Blame/Credit: unresolved Sgt’s Appraisal of Accident from Lieutenant’s Perspective Eagle 1-6 Assist Desire: LT Satisfied: False Appraisal

26 Child Healthy:False Accident Blame: unresolved Assist Eagle 1-6:False Eagle 1-6 Assist Desire: LT Satisfied: False Child-Healthy Desire: SGT Satisfied: False Distress: 80 Coping

27 Child Healthy:False Accident Blame: unresolved Assist Eagle 1-6:False Eagle 1-6 Assist Desire: LT Satisfied: False Distress: 80 Sgt’s Own Perspective Coping Child-Healthy Desire: SGT Satisfied: False

28 Child Healthy:False Accident Blame: unresolved Assist Eagle 1-6:False Eagle 1-6 Assist Desire: LT Satisfied: False Distress: 80 Child-Healthy Desire: SGT Satisfied: False Probability: 75%  Distress: 80 Coping Make Amends Get Medevac Responsibility:LT Problem-Focused Coping: Form intention to help Boy

29 Child Healthy:False Accident Blame: MOM Assist Eagle 1-6:False Eagle 1-6 Assist Desire: LT Satisfied: False Distress: 80 Child-Healthy Desire: SGT Satisfied: False Probability: 75%  Distress: 80 Coping Make Amends Get Medevac Responsibility:LT Shift Blame Emotion-Focused Coping: Blame Mother

30 Child Healthy:False Accident Blame: unresolved Assist Eagle 1-6:False Eagle 1-6 Assist Desire: LT Satisfied: False Distress: 80 Child-Healthy Desire: SGT Satisfied: False Probability: 75%  Distress: 80 Coping Make Amends Get Medevac Responsibility:LT Shift Blame Personality

31 MRE: Leadership Training

32 Architectural Implications Emotion as central control construct –Planning (inform course-of-action selection) –NLU (inform reference resolution) –Dialogue (prompt dialogue initiative) –NLG (biases sentence generation strategies) –Non-verbal expression

33 General Implications Emotion and Reflection –Appraisal is form of self-reflection / focus of attention Emotion as decision-making –Generalization of decision-theory –More to the world than probabilities and utilities Emotion and plausible reasoning –Emotion-focused coping motivate preference/beliefs –Attempt to construct coherent motivational explanation –Non-rational but adaptive?

34 General Implications Emotion and Learning –Focus learning on “emotionally salient” events –Appraisal variables as features / case indexes

35 Conclusion Emotion is form of information processing –Arguable adaptive Juggling competing goals and commitments Focusing cognitive resources Enforcing coherence –Arguable unexplored by cognitive systems

36

37 Focus=1 Lt: U9 “Secure a landing zone” Committed(lt,7), 7 authorized, Obligation(sgt,U9) Sgt: U10 “First we should secure the assembly area” Disparaged(sgt, 7), endorsed(sgt,2) Lt: U11“Secure the area” Committed(lt,2), 2 authorized, Obligation(sgt,U11) Sgt: U12 “Yes sir” Committed(sgt,2), Push(2) Goal7:Announce(2,{1sldr,2sldr,3sldr,4sldr}) Goal8: Start-conversation(sgt, {1sldr,2sldr,…},2) Goal8  Sgt: U13 “Squad leaders listen up!” Goal7  Sgt: U14 “I want 360 degree security” Push(3) Goal9:authorize 3 Goal9  Sgt: u15“1st squad take 12-4” Committed(sgt,3), 3 authorized Pop(3), Push(4) Goal10: authorize 4 Goal10  Sgt: u16“2nd squad take 4-8” Committed(sgt,4), 4 authorized Pop(4) … A10: Squads move A10: grounds U13-U18,… ends conversation about 2, realizes 2 Pop(2), Push(7) Dialogue Example: Sgt’s Behavior Render Aid Secure Area Secure 12-4 Secure 8-12 Secure Accident Secure 4-8 Squads in area A=Lt, R=Sgt A=Sgt, R=1sldr A=Sgt, R=2sldr A=Sgt, R=4sldr A=Sgt, R=3sldr Area Secure Decomposition Secure LZ A=Lt, R=Sgt 7

38 Appraisal as a mediating variable –Direct mappings (e.g. Hayes-Roth personality model) –Indirect mappings Direct: More links, No insight on how to map Indirect: more constrained. More modular More than a theory of emotion World state Beliefs Desires Personality Behavior World state Beliefs Desires Personality Behavior Appraisal Variables

39 Mediating Variable Appraisal Mediates Personality Personality Variable  Appraisal Variables  Behavior e.g Extroversion   Control   Hope  Penley & Tomaka (2002) Appraisal Mediates Culture Culture Variable  Appraisal Variables  Behavior e.g. Uncertainty avoidance   Threat   Fear  Kupperbusch et al

40 Mediating variables Coping mediated by appraisal –Undesirable & Controllable  Distress  Problem directed coping –Undesirable & Uncontrollable  Distress  Emotion directed coping


Download ppt "Jonathan Gratch USC Institute for Creative Technologies Joint work with Stacy Marsella USC Information Sciences Institute Jonathan Gratch USC Institute."

Similar presentations


Ads by Google