Evidence for Relative Position Coding in the Posterior Fusiform Gyrus Mark D. Lescroart, Kenneth Hayworth, Irving Biederman Neuroscience Program University of Southern California Presented at the Society for Neuroscience Conference November 6, 2007

Relations are easy and fast “This Way” by Robert Hague

Intro Shape Demo

Unlabeled Feature Hierarchy … V1 Anterior Infero-temporal Cortex / Posterior Fusiform Gyrus Ventral stream hierarchy

New Object S1 S2 Identical Translated Relation Translated + Relation or...or… Trial Types: Experiment 1

First Trial Demo 3.3˚ Original airplane position (200 ms S1) (200 ms S2) (500 ms ISI) ~5˚ of Visual Angle

Possible S1 Arrangements Eight possible starting (S1) arrangements:

Example Trial - Trans+Rel Translation + Relation Change Trial 4.6˚ Original airplane position

Example Trial - Ident Baseline: Identical Trial

Example Trial - New Object Task: Detect New Object Trial

Relation (swap) Participation1 - Trans 1

Relation (swap) Participation2 - Ident 2

Relation (swap) Participation3 - Trans 3

Participation4 - NewObj 4

Relation (swap) Participation5 - Rel 5

New Object S1 S2 Identical Translated Relation Translated + Relation or...or… Trial Types: Experiment 1

(S1) Predictions of various models: Large receptive fields, unlabeled feature list (e.g. the MIT “Standard Model”) (Change in Representation) IdenticalTranslatedRelation New Object Translated + Relation

(S1) Predictions of various models: Large receptive fields, unlabeled feature list (e.g. the MIT “Standard Model”) Smaller, retinotopically-specific receptive fields (Change in Representation) IdenticalTranslatedRelation New Object Translated + Relation

(S1) Predictions of various models: Large receptive fields, unlabeled feature list (e.g. the MIT “Standard Model”) Smaller, retinotopically-specific receptive fields (Change in Representation) Explicitly encoded relations / labeled feature list IdenticalTranslatedRelation New Object Translated + Relation

Behavioral Results: Experiment 1 false alarmsmisses

Region of Interest: Bilateral Posterior Fusiform Gyrus Defined by a standard localizer: -= We took the most significant voxels (P bonf <.01) in the most anterior portion of the posterior fusiform gyrus

MRI Results: Experiment 1 Posterior Fusiform Gyrus

MRI Results: Experiment 1 Relation > Translate p<.03 Trans+ Relation > Translated p<.01 Trans+ Relation > Retinotopic Prediction p<.01 Relation > Trans+ Relation n.s. Posterior Fusiform Gyrus * *

Example T1 trial (in 45 degrees reference frame) Other Changes: o Shortened stimulus duration (100 ms) o White-on-black line drawing instead of grayscale images

T2 T1 T2+R T1+R A New Object trial can be any of these rearrangements with one of the objects changed. Trial Types: Experiment 2 Ident

Prediction from Cortical Magnification: Foveal over- representation (Cortical Magnification) Explicitly encoded relations / labeled feature list (S1) IdenticalT1 T2T1+RT2+R (Change in Representation)

MRI Results: Experiment 2 T1 + Relation > T1 p<.01 T2 + Relation > T2 p<.01 T1 + Relation > T2 + Relation n.s. Note: Trend of T1 > T2 and T1+R > T2+R is close to significant * * Posterior Fusiform Gyrus

Could relations be explained by global or inter-object features?

Ident+GF Trans+GF Rel T+R NewObj One of above trial types where one object changes identity S1 S2

Trial Types: Experiment 3

Standard Model Predictions C2 Correlation taken as a measure of similarity of representation The model comes close to predicting the initial results (though the translation condition is probably too high)

Standard Model Predictions C2 Correlation taken as a measure of similarity of representation The model comes close to predicting the initial results (though the translation condition is probably too high) But with the addition of the texture backgrounds, the OPPOSITE effect is predicted. Thus if global features are driving the BOLD response, the changes in Experiment 3 ought to drastically change the results

Relation > Translate+GF p<.01 Trans+ Relation > Translate+GF p=.01 Relation > Trans+ Relation n.s. MRI Results: Experiment 3 * * Posterior Fusiform Gyrus

(S1) Predictions of various models: Large receptive fields, unlabeled feature list (e.g. the MIT “Standard Model”) Smaller, retinotopically-specific receptive fields Explicitly encoded relations / labeled feature list IdenticalTranslatedRelation New Object Translated + Relation (Change in Representation)

Conclusions Relative position changes have more of an effect on the neural representation in the posterior fusiform gyrus than absolute position changes This effect can not be explained by eccentricity, eye movements, or global features

Acknowledgements Co-Authors: Ken Hayworth Dr. Irving Biederman Special Thanks to: Dr. Xiaomin Yue Jiye Kim Xiaokun Xu Dave Berg Funded by: NSF Grants BCS , , and to I.B.

終

Extra Slides

Low-frequency / global shape features Inter- object features (nothing) Scene outline (nothing) Could global features explain the greater release from adaptation in the relation- change conditions? (nothing)

Experiment 3: Adding Global Feature Changes

Measuring Similarity of Representation in the Brain Response to two different stimuli, presented close together in time Response to two of the same stimuli, presented close together in time

(S1) Predictions of various models: Attention Shifts: Saliency driven Attention Shifts: Position order driven (Change in Representation) Explicitly encoded relations / labeled feature list IdenticalTranslatedRelation New Object Translated + Relation (A-B-B-A)(A-B-A-B) (A-B-A-B)(A-B-A-B)

Subject presses key when ready to start trial 33ms scene presentation 100ms mask Variable blank delay (0ms to 133ms) Subject writes down the names of both objects and their relative spatial relation Fast Two-Object Naming Experiment

Fast Naming Results Stimulus 33ms Mask 100ms SOA (Chance)

Behavioral Results: Experiment 1

Behavioral Results: Experiment 2

Behavioral Results: Experiment 3

MRI Results: Experiment 2

MRI Results: Experiment 3

Stimulus Parameters 2.1 ˚ 3.3˚ 2.3˚ 3.3˚ S1 Pos S1 Pos 4.6˚

Example T1 trial (in 0 degrees reference frame)

Example T2 trial (in 0 degrees reference frame)

Example T2 trial (in 90 degrees reference frame)

Example T1 trial (in 90 degrees reference frame)

Example T2 trial (in 45 degrees reference frame)