Attention and emotion: From data to conceptual issues Luiz Pessoa Department of Psychology University of Maryland, College Park

Background 1990s: work showing limitations of visual processing and the need for attention  Change blindness  Attentional blink

Background Processing of emotion-laden information is prioritized  Independent of awareness R L Morris et al. (1998) Whalen et al. (1998)

LeDoux “Automatic” Processing Amygdala

Research goal Understand the role of attention and awareness during the processing of emotional visual items  Employ strong attentional manipulations  Evaluate awareness with Signal Detection Theory

Role of spatial attention Is activity evoked by emotional faces automatic? OR Does activity evoked by emotional faces require attention?

200 ms Same/different Difficult: 64% correct Not drawn to scale 200 ms Attended Faces Unattended Faces Spatial attention Male/female Easy: 91% correct

Attention is required for the expression of valence (N = 21) Strong valence X attention interaction: Effect of valence depends on attention Fear UNATT Happy UNATT Neutral UNATT Happy ATT Neutral ATT Fear ATT Right Amygdala Response Amplitude Seconds Pessoa et al. (2002): PNAS X LR

Emotional perception requires attention Attention parametrically manipulated within the same task Affective significance enhanced via conditioning Lim et al. (2008): Neuropsychologia Task: find X EasyHard Pessoa et al. (2005): Neuroimage Hsu and Pessoa (2007): Neuropsychologia

Role of visual awareness Pessoa et al. (2005): Emotion Target Mask Yes/No 1-3 scale Confidence?

67 ms 33 ms Visual awareness: Signal Detection

67 ms Y = -4 AWARE LR 33 ms Y = -4 UNAWARE LR Amygdala responses VS. Pessoa et al. (2006): Cerebral Cortex

Many participants can detect fearful faces even at 17 ms Behavioral results: Individual differences Szczepanowski and Pessoa et al. (2007): Journal of Vision 17 ms

Fear stimulus > Neutral stimulus Amygdala Pessoa et al. (2006): Cerebral Cortex 67 ms “Normals” N = 19 Y = -4 AWARE LR “Detecters” N = 8 Y = -6 AWARE LR 33 ms Y = -4 UNAWARE LR Y = -6 AWARE LR x.

Role of temporal attention/awareness ms Lim, Padmala, and Pessoa (2009): PNAS 2 s T2 T1 CS+ vs. CS–

Attentional blink: Behavior (N = 30) Enhanced perception of CS+: Reduced blink T1 T2 CS+ CS– Building or House?

Role of attention/awareness... Parahippocampal gyrus T2

Miss trials... T2 Parahippocampal gyrus

Role of attention/awareness Miss trials: no differences observed between CS+ and CS- trials Visual ctx % signal change Amygdala Time % signal change CS+ CS-

Conceptual issues

Impasse While a great deal has been learned about the extent and limits of affective visual processing, two camps have opposing and entrenched views

Impasse While a great deal has been learned about the extent and limits of affective visual processing, two camps have opposing and entrenched views Capacity-limited Capacity-Unlimited

Impasse While a great deal has been learned about the extent and limits of affective visual processing, two camps have opposing and entrenched views Capacity-limited Capacity-Unlimited

Not too surprising… Emotional stimuli are sufficiently potent that they exhibit a host of properties that do not appear to occur with neutral items  They are processed when unattended Affective processing is subject to capacity limitations, as revealed by several experimental manipulations  Attentional blink

Impasse: will it go away? Advocates of limited processing can claim that processing resources have not been consumed  “If the manipulation were stronger, the impact of affective items would go away…”

Impasse Showing that the emotional effect has disappeared is always subject to the “null problem”  Arguing for the absence of an effect

Power vs. strength of manipulation Fear unatt Happy unatt Neutral unatt Happy att Neutral att Fear att Right Amygdala Seconds Response Amplitude Left Amygdala Seconds Pessoa et al. (2002): PNAS X

processing resources performance Processing resources Given the limited capacity of mental processes, performance is impaired if demands are greater than available resources Easy/efficient Hard/inefficient Norman and Bobrow (1975)

Processing resources processing resources performance processing resources performance “automatic” Nakayama and Joseph (1998)

Processing resources processing resources performance processing resources performance Capacity limitation Dual-tasks Nakayama and Joseph (1998)

Processing resources Moors and De Houwer (2006): Every process is uncontrolled, efficient, unconcious, and fast

Processing resources Moors and De Houwer (2006): Every process is uncontrolled, efficient, unconcious, and fast, to some degree… Relative to what? Affective processing: relative to neutral not enough  Fine comparisons needed (e.g., abrupt onsets, search, etc.)  Broad set of comparison tasks

Two camps Capacity-limited Capacity-Unlimited

Preattentive-attentive model t1t1 t2t2 Stage 1: Preattentive Stage 2: Attentive boundary

Preattentive-attentive model Some features are processed pre-attentively in virtue of the fact that they are optimally matched to properties of the early visual system (e.g., orientation) Affective processing: Sub-cortical pathway  Superior colliculus  pulvinar  amygdala

Dynamic model Multiple interactive “stages”

Dynamic model Processing is not pre-attentive or attentive, but a gradient of processing efficiency is hypothesized to exist Gradient based on the properties of early visual areas But critically, gradient is dynamically configured based on task demands Configuring is suggested to depend on parietal and frontal cortex

Dynamic model Multiple “gates” Variable permeability Less susceptible to capacity limitations More susceptible to capacity limitations “bottlenecks”

Dynamic model Hierarchical and “short-cut” connections

Multiple waves Initial processing of visual information proceeds simultaneously along parallel channels “Multiple waves” of activation across visual cortex and beyond The multiple waves are engaged dynamically based on task requirements

Subcortical processing

Pessoa and Adolphs, Nat. Rev. Neurosci (2010) “passive” “integrative”

Subcortical processing Pessoa and Adolphs, Nat. Rev. Neurosci (2010)

Processing architecture and attention Task 1 Task 2

Collaborators  Ralph Adolphs  Jan Engelmann  Shruti Japee  Shen-Mou Hsu  Seung-Lark Lim  Srikanth Padmala  Remik Szczepanowski  Leslie Ungerleider National Institute of Mental Health emotioncognition.org