1. Evidence for Relative Position Coding in the Posterior Fusiform Gyrus Mark D. Lescroart, Kenneth Hayworth, Irving Biederman Neuroscience Program University of Southern California Presented at the Society for Neuroscience Conference November 6, 2007

2. Relations are easy and fast “This Way” by Robert Hague

3. Intro Shape Demo

4. Unlabeled Feature Hierarchy … V1 Anterior Infero-temporal Cortex / Posterior Fusiform Gyrus Ventral stream hierarchy

5. New Object S1 S2 Identical Translated Relation Translated + Relation or...or… Trial Types: Experiment 1

6. First Trial Demo 3.3˚ Original airplane position (200 ms S1) (200 ms S2) (500 ms ISI) ~5˚ of Visual Angle

7. Possible S1 Arrangements Eight possible starting (S1) arrangements:

8. Example Trial - Trans+Rel Translation + Relation Change Trial 4.6˚ Original airplane position

9. Example Trial - Ident Baseline: Identical Trial

10. Example Trial - New Object Task: Detect New Object Trial

11. Relation (swap) Participation1 - Trans 1

12. Relation (swap) Participation2 - Ident 2

13. Relation (swap) Participation3 - Trans 3

14. Participation4 - NewObj 4

15. Relation (swap) Participation5 - Rel 5

16. New Object S1 S2 Identical Translated Relation Translated + Relation or...or… Trial Types: Experiment 1

17. (S1) Predictions of various models: Large receptive fields, unlabeled feature list (e.g. the MIT “Standard Model”) (Change in Representation) IdenticalTranslatedRelation New Object Translated + Relation

18. (S1) Predictions of various models: Large receptive fields, unlabeled feature list (e.g. the MIT “Standard Model”) Smaller, retinotopically-specific receptive fields (Change in Representation) IdenticalTranslatedRelation New Object Translated + Relation

19. (S1) Predictions of various models: Large receptive fields, unlabeled feature list (e.g. the MIT “Standard Model”) Smaller, retinotopically-specific receptive fields (Change in Representation) Explicitly encoded relations / labeled feature list IdenticalTranslatedRelation New Object Translated + Relation

20. Behavioral Results: Experiment 1 false alarmsmisses

21. Region of Interest: Bilateral Posterior Fusiform Gyrus Defined by a standard localizer: -= We took the most significant voxels (P bonf <.01) in the most anterior portion of the posterior fusiform gyrus

22. MRI Results: Experiment 1 Posterior Fusiform Gyrus

23. MRI Results: Experiment 1 Relation > Translate p<.03 Trans+ Relation > Translated p<.01 Trans+ Relation > Retinotopic Prediction p<.01 Relation > Trans+ Relation n.s. Posterior Fusiform Gyrus * *

24. Example T1 trial (in 45 degrees reference frame) Other Changes: o Shortened stimulus duration (100 ms) o White-on-black line drawing instead of grayscale images

25. T2 T1 T2+R T1+R A New Object trial can be any of these rearrangements with one of the objects changed. Trial Types: Experiment 2 Ident

26. Prediction from Cortical Magnification: Foveal over- representation (Cortical Magnification) Explicitly encoded relations / labeled feature list (S1) IdenticalT1 T2T1+RT2+R (Change in Representation)

27. MRI Results: Experiment 2 T1 + Relation > T1 p<.01 T2 + Relation > T2 p<.01 T1 + Relation > T2 + Relation n.s. Note: Trend of T1 > T2 and T1+R > T2+R is close to significant * * Posterior Fusiform Gyrus

28. Could relations be explained by global or inter-object features?

29. Ident+GF Trans+GF Rel T+R NewObj One of above trial types where one object changes identity S1 S2

30. Trial Types: Experiment 3

31. Standard Model Predictions C2 Correlation taken as a measure of similarity of representation The model comes close to predicting the initial results (though the translation condition is probably too high)

32. Standard Model Predictions C2 Correlation taken as a measure of similarity of representation The model comes close to predicting the initial results (though the translation condition is probably too high) But with the addition of the texture backgrounds, the OPPOSITE effect is predicted. Thus if global features are driving the BOLD response, the changes in Experiment 3 ought to drastically change the results

33. Relation > Translate+GF p<.01 Trans+ Relation > Translate+GF p=.01 Relation > Trans+ Relation n.s. MRI Results: Experiment 3 * * Posterior Fusiform Gyrus

34. (S1) Predictions of various models: Large receptive fields, unlabeled feature list (e.g. the MIT “Standard Model”) Smaller, retinotopically-specific receptive fields Explicitly encoded relations / labeled feature list IdenticalTranslatedRelation New Object Translated + Relation (Change in Representation)

35. Conclusions Relative position changes have more of an effect on the neural representation in the posterior fusiform gyrus than absolute position changes This effect can not be explained by eccentricity, eye movements, or global features

36. Acknowledgements Co-Authors: Ken Hayworth Dr. Irving Biederman Special Thanks to: Dr. Xiaomin Yue Jiye Kim Xiaokun Xu Dave Berg Funded by: NSF Grants BCS 04-20794, 05-31177, and 06-17699 to I.B.

37. 終

38. Extra Slides

39. Low-frequency / global shape features Inter- object features (nothing) Scene outline (nothing) Could global features explain the greater release from adaptation in the relation- change conditions? (nothing)

40. Experiment 3: Adding Global Feature Changes

41. Measuring Similarity of Representation in the Brain Response to two different stimuli, presented close together in time Response to two of the same stimuli, presented close together in time

42. (S1) Predictions of various models: Attention Shifts: Saliency driven Attention Shifts: Position order driven (Change in Representation) Explicitly encoded relations / labeled feature list IdenticalTranslatedRelation New Object Translated + Relation (A-B-B-A)(A-B-A-B) (A-B-A-B)(A-B-A-B)

43. Subject presses key when ready to start trial 33ms scene presentation 100ms mask Variable blank delay (0ms to 133ms) Subject writes down the names of both objects and their relative spatial relation Fast Two-Object Naming Experiment

44. Fast Naming Results Stimulus 33ms Mask 100ms SOA (Chance)

45. Behavioral Results: Experiment 1

46. Behavioral Results: Experiment 2

48. Behavioral Results: Experiment 3

50. MRI Results: Experiment 2

51. MRI Results: Experiment 3

52. Stimulus Parameters 2.1 ˚ 3.3˚ 2.3˚ 3.3˚ S1 Pos S1 Pos 4.6˚

53. Example T1 trial (in 0 degrees reference frame)

54. Example T2 trial (in 0 degrees reference frame)

55. Example T2 trial (in 90 degrees reference frame)

56. Example T1 trial (in 90 degrees reference frame)

57. Example T2 trial (in 45 degrees reference frame)