Cortical Representations of Attention and Salience James A. Mazer Department of Neurobiology Yale School of Medicine Theoretical and Experimental Approaches to Auditory and Visual Attention Banbury 2008

The visual scenes can be complex during natural vision Cayo Santiago, MH, 2007

Neural Representations of Attention and Salience 1.Feature-based attention in area V4 –labeled lines or matched filters? –(monkey single-unit) 2.Maintaining spatial attention during eye movements –spatiotopic or retinotopic maps for spatial attention? –(human psychophysics)

Attention and salience maps elevation azimuth orientation spatial frequency phase color/wavelength Integration Rules ??? search target:

elevation azimuth orientation Attention and salience maps spatial frequencyphase Integration Rules ??? search target: color/wavelength feature-based attention

elevation azimuth orientation Attention and salience maps Integration Rules ??? spatial cue: target on right spatial frequencyphase color/wavelength spatial attention

elevation azimuth orientation Attention and salience maps Integration Rules ??? spatial cue: target on left spatial frequencyphase color/wavelength spatial attention Koch, Itti, Heeger & Reynolds …

IT V4 V1 1. “Feature-based” attention in area V4 V2 - essential form processing relay station - known anatomical and physiological target for top-down signals

RSVP: (slow) Rapid Search Visual Presentation search target stimulus stream

RSVP: (slow) Rapid Search Visual Presentation search target stimulus stream +

RSVP: (slow) Rapid Search Visual Presentation search target stimulus stream +

RSVP: (slow) Rapid Search Visual Presentation search target stimulus stream +

RSVP: (slow) Rapid Search Visual Presentation search target stimulus stream +

RSVP: (slow) Rapid Search Visual Presentation search target stimulus stream +

RSVP: (slow) Rapid Search Visual Presentation search target stimulus stream +

RSVP: (slow) Rapid Search Visual Presentation search target stimulus stream +

RSVP: (slow) Rapid Search Visual Presentation search target stimulus stream +

RSVP: (slow) Rapid Search Visual Presentation search target stimulus stream + match!

RSVP: (slow) Rapid Search Visual Presentation search target stimulus stream + match!

Tuning during RSVP: Spike Triggered Average tt tt tt tt tt tt tt Response (Hz)‏ Orientation (deg)‏

Tuning during RSVP: Spike Triggered Average tt tt tt tt tt tt tt Response (Hz)‏ Orientation (deg)‏

Tuning during RSVP: Spike Triggered Average tt tt tt tt tt tt tt Response (Hz)‏ Orientation (deg)‏

Tuning during RSVP: Spike Triggered Average tt tt tt tt tt tt tt Response (Hz)‏ Orientation (deg)‏

Tuning during RSVP: Spike Triggered Average tt tt tt tt tt tt tt Response (Hz)‏ Orientation (deg)‏

Tuning during RSVP: Spike Triggered Average tt tt tt tt tt tt tt Response (Hz)‏ Orientation (deg)‏

Interim Summary: Feature-based attention V4 feature-based attention affects: –shifts in preferred stimuli –orientation-dependent gain changes changes in preferred stimulus could facilitate target detection, but not in any simple manner (some +, some -)‏ modulation is maximal in a sub-population of broadly tuned V4 neurons, suggesting multiple intermingled populations within V4 –perhaps reflecting modulated and veridical subpopulations… elevation azimuth orientation spatial frequency phase color/wavelength

2. Maintaining spatial attention across eye movements

Spatial cueing 101 Posner (1980) and many others...

Spatial cueing 101 Posner (1980) and many others... valid cueinvalid cue

Spatial cueing 101 Posner (1980) and many others... valid cueinvalid cue valid invalid faster more accurate more sensitive etc…

Neural representation of the cue?

Retinotopic (eye-centered)

Neural representation of the cue? Spatiotopic (head-centered) Retinotopic (eye-centered)

The representation of spatial attention Spatial attention survives intervening saccades Spatial attention is transiently maintained in retinotopic coordinates even when it hurts... The native representation of spatial attention appears to be retinotopic Simple model: retinotopy + recurrence... elevation azimuth

Conclusions  feature-based attention –feature-based in V4 can alter both gain and preferred stimulus –orientation tuning changes are “correlated” with behavioral goals  maintaining spatial attention across eye movements –saccade planning and spatial cueing are dissociable (anti-premotor) –spatial attention is intrinsically retintopic –spatiotopic representations required active, top-down updating or remapping under voluntary control

Acknowledgements Mazer Lab Jonathan Touryan Monica Cano Vinas Julie Golomb Matt Krause Marvin Chun (Yale Psychology)