Neural mechanisms of feature- based attention Taosheng Liu

What is attention? “Everyone knows what attention is. It is the taking possession by the mind in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought.” -William James (1890)‏ Types of visual attention –Overt attention –Covert attention Spatial Feature-based Object-based

Attention and the brain Effects vs. control

Outline The effect of feature-based attention on visual cortex –How does attention modulate sensory representations? The control of feature-based attention –What is the source of control and how is control implemented? Attention and object recognition

The effect of FB attention to motion Treue & Martinez-Trujillo, 1999, Nature Response Attend ‘up’ Attend ‘down’ Questions: Does feature-based attention modulate neuronal subpopulations in the attended location? If so, how does it correlate with behavior? MT

Response Attend ‘up’ Attend ‘down’ upward preferring units More adaptation for a upward test stimulus when attending ‘up’ vs. ‘down’ Response downward preferring units Use adaptation to assess feature selectivity

fMRI adaptation A voxel contains many neurons. fMRI adaptation can assess feature selectivity within a voxel.

Adapting stimulus play demo +20°-20°

Behavior: tilt aftereffect (n=8)‏ +20°-20° 0°..... Adapter (4 s)‏ Test (0.5 s)‏ 1 s … Pre-adaptation (40 s)‏ Attend +20 Attend -20 Attend +20 Attend -20

fMRI adaptation protocol..... Adapter (4 s)‏Test (1 s)‏ 1 s 1.2 s Attended Unattended Blank … Pre-adaptation (40 s)‏ … Task inside the scanner: report the orientation of the test stimulus.

fMRI details Siemens 3T Allegra Surface coil 21 coronal/oblique slices 3 mm isotropic voxels TE = 30 ms, FA = 75º TR = 1.2 s Bite bar to minimize head motion

Surface reconstruction and retinotopic mapping

Retinotopic mapping and localizer real data (TL)‏

fMRI response to the test stimulus Unattended Attended adapter test

Attention modulation index R attn – R unattn R attn + R unattn

Correlation between behavioral and imaging results

A model relating psychophysical and imaging data neutral attended Preferred orientation (deg)‏ Shift in preferred orientation Preferred orientation (deg)‏ Dragoi et al, 2000, Preferred orientation (deg)‏ Psychophysics fMRI Neural response

Summary & conclusion Feature-based attention enhances activity of neuronal subpopulations when the attended and unattended features are processed in the same retinotopic region. –Attentional modulation of orientation-selective fMRI response adaptation. –Attentional modulation constant across visual areas, suggesting a feed-forward mechanism. –Significant correlation between TAE and AMI only in V1. Liu etal, 2007, Neuron

The control of feature-based attention Components of attentional control –Disengage/shift –Engage/maintain Goal: –Separate different components –Feature-based attention

Task and design responseMotionColorinstruction button2Green‘hold’ button1Red‘shift’

Sustained effect for motion R SPL/IPL % signal change Time (sec)‏ color to motion MT+: effects of attention for motion. FEF, SPL/IPL: sustained attentional control for motion. motion to color hold motion hold color

Transient shift activity color to motion motion to color hold motion hold color Precu, IPS, PCG: transient control of attention shift.

Summary Effects of attention: –MT+ (motion) and V4 (color)‏ Attentional control: –Transient control: disengage/shift (superior parietal lobule, left intra-parietal sulcus, left pre-central gyrus). –Sustained control: engage/maintain (frontal eye fields, superior- inferior parietal lobule for motion; superior frontal gyrus for color). Liu etal, 2003, Cerebral Cortex

Current and future plans Attentional control within feature dimensions –What are the ‘shift’ regions? –What are the ‘hold’ regions?--attentional priority

The representation of attentional priority Spatial attention –Higher areas with a spatiotopic map send feedback signals Feature-based attention –Are there neurons that encode the attended direction in higher areas? FEFLIP

Decoding of brain activity Kamitani & Tong (2007)‏ Classifier scheme Classifier can reliably decode orientation information in early visual cortex

Learning sequence of views of three-dimensional objects: The effect of temporal coherence on object memory

How do we recognize shapes? Temporal association: object views appearing close in time are associated. Wallis & Bulthoff (1999)‏

Harman & Humphrey (1999)‏ 7 views x 1 s/view x 3 repeats No accuracy effects Attention? Effort? ???

Exp 1 - replication stimuli

Exp 1 - method

Exp 1 - results

Exp 2: test novel views Test views 1,3,5,7

Exp 3 – method

Exp 3 - results

Exp 4 Encoding task: preference rating “rate how much you like each sequence on a 3-point scale”

Exp 4 - results

Summary RR always the worst –temporal association works SS never exceeds SR –temporal vs. spatiotemporal coherence SS depends on study time and intention –potential confound