Multiple Information Pathways in the Expression of Conversational Affect S YNCHRONY & S YMMETRY S YMPOSIUM APS, M AY, 2016 A LLISON G RAY, T IMOTHY R.

Slides:

Advertisements

Similar presentations

Affective Facial Expressions Facilitate Robot Learning Joost Broekens Pascal Haazebroek LIACS, Leiden University, The Netherlands.

Advertisements

The world of emotion is two-dimensional,.. or is it? Etienne B. Roesch (Univ. Geneva) Johnny R. J. Fontaine (Univ. Ghent) Klaus R. Scherer (Univ. Geneva)

Matthias Wimmer, Ursula Zucker and Bernd Radig Chair for Image Understanding Computer Science Technische Universität München { wimmerm, zucker, radig

Emotional Transformation Psalm 139 Emotional Transformation Psalm 139.

Emotion Michael Hoerger. Emotion Affect (af-ekt): the psychologist’s word for emotion, especially outwardly expressed emotion Affect (af-ekt): the psychologist’s.

Age Differences in Emotion Recognition of Briefly Presented Faces Lisa Emery, Kory Morgan, Kaitlyn Pechanek & Caitlin Williams Reprints may be obtained.

1 Emotion perception in old age and dementia Louise Phillips, Vasiliki Orgeta & Clare Scott School of Psychology, University of Aberdeen. Acknowledgements:

Measuring emotions in an intergroup context and beyond Dr Roger Giner-Sorolla Department of Psychology Presentation given at the Research Methods Festival,

GERG Santorini, June, 2004 “Ekman versus Scherer” or “What is important to recognize/express emotion?” (Selected GERG studies directed by Susanne.

Emotion. The heart has reasons that reason does not recognize -- Pascal Reason is and ought to be the slave of passion -- Hume Are Emotions Necessary.

Emotion, Intuition and “Hot” Thinking I.Psychological Perspectives on Emotion  the role of thinking in emotion II.Studying Emotion III.Blink: Thin Slicing.

Ch 4: Perceiving Persons Part 1: Sept. 17, Social Perception Get info from people, situations, & behavior – We make quick 1 st impressions of people.

EKMAN’S FACIAL EXPRESSIONS STUDY A Demonstration.

Emotions: Emotions: Hampson, E., van Anders, S. M., & Mullin, L. I. (2006). A female advantage in the recognition of emotional facial expressions: test.

Recognizing Emotions in Facial Expressions

Non-Verbal Communication

Facial Emotion Recognition and Social Functioning in Children With and Without Attention-Deficit Hyperactivity Disorder Marissa Reynolds The Pennsylvania.

Aging and the use of emotional cues to guide social judgments Louise Phillips, Gillian Slessor & Rebecca Bull University of Aberdeen.

Types of Nonverbal Communication and Body Language

Chapter 10 Conducting & Reading Research Baumgartner et al Chapter 10 Qualitative Research.

Categorizing Emotion in Spoken Language Janine K. Fitzpatrick and John Logan METHOD RESULTS We understand emotion through spoken language via two types.

Components of Emotion: Facial expressions Physiological factors (e.g., heart rate, hormone levels) Subjective experience/feelings Cognitions that may elicit.

Nonverbal Communication

It’s what you say… and what you don’t Nonverbal Messages in Communication.

From: Carter R (1998) Mapping the Mind. Seven Dials, London.

Maria Neophytou Communication And Internet Studies ENG270 – English for Communication Studies III

A Study in Cross-Cultural Interpretations of Back-Channeling Behavior Yaffa Al Bayyari Nigel Ward The University of Texas at El Paso Department of Computer.

Method Participants Participants were 68 preschoolers, between the ages of 29 and 59 months of age. The sample was comprised of 32 male participants and.

Emotion Psychology Introduction Emotions are a mix of: Emotions are a mix of: Physiological arousal of some sort Physiological arousal of some sort.

Consider the target audience. Companies tailor their messages to a specific age, ethnicity, gender, or income bracket. Companies tailor their messages.

E Cognitive Science and Educational Technology I Jan L. Plass, ECT.

 Cognitive appraisal patterns the same for each emotion across cultures.

Within Subjects Designs AKA Repeated Measures Design Remember matched groups design? –Allowed more power if you matched on correct variable –Within Subjects.

Nonverbal communication

Method Introduction Results Discussion Different Neurocognitive Abilities Moderate the Relationship between Affect Perception and Community Functioning.

BZUPAGES.COM Organizational Behaviors Presented to: Sir Ahmad Tisman Pasha Presented by: Fiaz Ahmad BIT Department of IT Institute of computing Bzu.

Unit 4: Emotions.

Social Development (Chapter 13) Lecture Outline: Emotional development The “self” and personality Temperament.

Non Verbal Communication. What Is Paralanguage? DEFINITION Paralanguage is the voice intonation that accompanies speech, including voice pitch, voice.

Theories of Emotions Module 41.

The many emotions across the adult life span, and how we interpret them Susan Charles, Ph.D.

Ekman’s Facial Expressions Study A Demonstration.

 Prototype: The best or average exemplar of a category.  In our case….an emotion that is typical of the category of emotions and represents all other.

Facial Expressions and Emotions Mental Health. Total Participants Adults (30+ years old)328 Adults (30+ years old) Adolescents (13-19 years old)118 Adolescents.

Acoustic Cues to Emotional Speech Julia Hirschberg (joint work with Jennifer Venditti and Jackson Liscombe) Columbia University 26 June 2003.

Feelings and Emotions. Angry What does it mean to be angry? What makes you angry or mad? Make a mad or angry face.

Discuss the extent to which cognitive and biological factors interact in emotion.

Interpreting Ambiguous Emotional Expressions Speech Analysis and Interpretation Laboratory ACII 2009.

EXAMPLES ABSTRACT RESEARCH QUESTIONS DISCUSSION FUTURE DIRECTIONS Very little is known about how and to what extent emotions are conveyed through avatar.

Infancy Emotional Development.  Roles of emotions  Help humans survive and adapt to their environment  Fear, smile  Guide and motivate human behavior.

Unique featural difference for happy and fear (in top down and middle out) and for happy and disgust (for bottom up): For fear, eyes are open and tense.

CH 6: Nonverbal Messages (slide 1) Chapter 6: Nonverbal Messages Copyright © 2013, 2009, 2006 Pearson Education, Inc. All Rights Reserved.

THE NATURE OF THE EMOTIONS. “Emotion” comes from the Latin “movere”, “to move”.

What is it? Details Look at the whole.  Form of communication ◦ Communicating without words ◦ Non-verbal ◦ Using facial expressions ◦ Gestures  Can.

Micro-Expressions. Micro-expressions are very brief facial expressions, lasting only a fraction of a second. They occur when a person either deliberately.

Communication. The sharing of a thought, idea or feeling Definition Different forms of communication: Reading Writing Listening Touching Seeing.

Emotion Knowledge in Maltreated Preschoolers

Emotions By: Loany Ham.

DON’T discuss your answers to these questions with the people around you What is something that you bought that you really enjoyed (or regretted)? What.

British face stimuli Egyptian face stimuli

Voluntary (Motor Cortex)

Using Body Language The Better Speaker Series 279.

Social context influence emotional language comprehension

Chapter 5: Nonverbal Communication

Emotion in Clinical Psychology

42.1 – Describe our ability to communicate nonverbally, and discuss gender differences in this capacity. Expressed Emotion Emotions are expressed on the.

Perceptual-Motor Deficits in Children with down syndrome: Implications for Intervention Study by: Naznin Virji-Babul, Kimberly Kerns, Eric Zhou, Asha.

Presentation transcript:

Multiple Information Pathways in the Expression of Conversational Affect S YNCHRONY & S YMMETRY S YMPOSIUM APS, M AY, 2016 A LLISON G RAY, T IMOTHY R. B RICK, S TEVEN M. B OKER

Outline Emotion in context, information pathways in a conversation Current Study: Parse apart information pathways Rater variability (no ground truth) How important are different information pathways?

What is the role of emotion? Emotion is… –“flexible, contextually bound” (Campos et al., 1994) Expressions of emotion are… –“directed at a receiver” (Russell et al., 2003) –“conceptual instances belonging to a single emotion category are highly variable” (Barrett, 2009)

How do we interpret emotional information? Visual: –Facial Expression –Gestures Audial: –Semantics (words, meaning of word) –Prosody (pitch inflection, cadence) Background information of conversation partner (if any) Banziger, Grandjean, & Scherer, 2009; Russell et al., 2003

How do we interpret emotional information? Visual: –Facial Expression –Gestures Audial: –Semantics (words, meaning of word) –Prosody (pitch inflection, cadence) Background information of conversation partner (if any) Banziger, Grandjean, & Scherer, 2009; Russell et al., 2003

Conditions 5 second video clips –selected & cut randomly from emotionally prompted conversations Audial: Visual: SoundGarbledSilent MovieFace, Semantic, and Prosody Face and ProsodyFace only StillSemantics and Prosody Prosody only

Hypotheses Informational Hypothesis: more information present will yield higher degree of consensus Assumption: All channels are in agreement Conflicting Information Hypothesis: When different pathways convey conflicting information, fewer pathways will lead more consensus.

Design X 31 words X 30 thin slice clips Randomly ordered clips and words Presented with PsychoPy Participants, N = 51, taken from UVA participant pool Peirce, 2007

Movie and Sound

Movie and Silent

Still and Sound

Movie and Garbled

Still and Garbled

Expression Rating Words surprise fear hidden excitement calm contempt relaxed compassion embarrassment sadness candid happiness joy guilt contentment anxiety stress open disappointment anger shame disguised hate peaceful pleasure despair irritation interest hurt shy disgust

Methods and Results SD Rating clip = b 0 + b 1 Video condition clip + b 2 Audio condition clip + e clip Removing person means Removing clip means

Results Still stimuli yielded more variability than Movie stimuli ( t = 39.41, p < ) sd = 1.50sd = 1.69

Results Garbled stimuli yielded more variability than Sound stimuli ( t = 18.16, p < ) sd = 1.64sd = 1.54

Results Silent stimuli yielded more variability than Sound stimuli ( t = 11.26, p < ) sd movie = 1.52sd movie = 1.49

Happy Words Positive words follow the pattern in the first hypothesis. Removing pathways increases variability and decreases consensus Face, Semantics, Prosody Face, Prosody Face Semantics, Prosody sd =

Happiness Ratings of ’Happiness’ roughly reflect the same pattern. More information leads to more consensus. Face, Semantics, Prosody Face, Prosody Face Semantics, Prosody sd =

Openness The breakdown of rater consensus is more exaggerated with openness, especially across the movie/still break. Face, Semantics, Prosody Face, Prosody Face Semantics, Prosody sd =

Interested Ratings of ‘Interested’ support the second hypothesis—when information is conflicting, more information leads to more variability. Prosody alone provides the most definitive information for ’Interested.’ Face, Semantics, Prosody Face, Prosody Face Semantics, Prosody sd =

Limitations Garbling process works better for some clips rather than others Not a diverse set of raters Long study so potential differences in ratings across course of study

Summary Removing pathways increases variability and decreases consensus Groups of words follow different patterns in breakdown of consensus –movie vs. still or sound vs. garbled vs. silent Some words do not follow this pattern at all Contact info: Allison Gray Thank you! –Funding: National Institute on Aging (R21-AG041035), National Science Foundation (BCS– ) and Max Planck Institute for Human Development. –Human Dynamics UVa –Undergraduate Research Assistants in HDL –Steven Boker –Tim Brick