1. Visual Neuron Responses This conceptualization of the visual system was “static” - it did not take into account the possibility that visual cells might change their response selectivity over time and it was firmly based in the classical notion of a receptive field –Logic went like this: if the cell is firing, its preferred line/edge must be present and… –if the preferred line/edge is present, the cell must be firing We will encounter examples in which these don’t apply! Representing boundaries and surfaces must be more complicated than simple edge detection! WHY??

2. What can a visual neuron “know” about the image? If a neuron is only an edge detector and/or only has a small receptive field, it can’t represent information about the relationship between the contents of its receptive field and other receptive fields elsewhere in the display.

3. What can a visual neuron “know” about the image? If a neuron is only an edge detector and/or only has a small receptive field, it can’t represent information about the relationship between the contents of its receptive field and other receptive fields elsewhere in the display. For example the famous 1961 Rosebowl hoax…no single person could know what the big picture showed

4. What can a visual neuron “know” about the image? If a neuron is only an edge detector and/or only has a small receptive field, it can’t represent information about the relationship between the contents of its receptive field and other receptive fields elsewhere in the display. Also the 2004 Harvard – Yale Game:

5. Visual Neuron Responses Edges are important because they are the boundaries between objects and the background or objects and other objects

6. Visual Neuron Responses Boundaries between objects can be defined by color rather than brightness

7. Visual Neuron Responses Boundaries between objects can be defined by texture

8. Visual Neuron Responses Boundaries between objects can be defined by motion and depth cues

9. Visual Neuron Responses Boundaries between objects can be defined by motion and depth cues

10. Feed-Forward and Feed-Back Processing in the Visual System

11. The Feed-Forward Sweep What is the feed-forward sweep?

12. The Feed-Forward Sweep The feed-forward sweep is the initial response of each visual area “in turn” as information is passed to it from a “lower” area Characteristics: – a single spike per synapse –no time for lateral connections –no time for feedback connections

13. The Feed-Forward Sweep The feed-forward sweep is the initial response of each visual area “in turn” as information is passed to it from a “lower” area What does it mean for an area to be “lower” or “higher”

14. The Feed-Forward Sweep Hierarchy of visual cortical areas defined anatomically Dorsal “where”/”how” Ventral “what” Notice the direct connection from SC to MT/V5

15. The Feed-Forward Sweep Hierarchy can be defined more functionaly The feed-forward sweep is the initial response of each visual area “in turn” as information is passed to it from a “lower” area Consider the latencies of the first responses in various areas

16. The Feed-Forward Sweep Thus the “hierarchy” of visual areas differs depending on temporal or anatomical features aspects of the visual system account for this fact: –multiple feed-forward sweeps progressing at different rates (I.e. magno and parvo pathways) in parallel M pathway is myelinated P pathway is not –signals arrive at cortex via routes other than the Geniculo-striate pathway (LGN to V1) Will be important in understanding blindsight

17. The Feed-Forward Sweep The feed-forward sweep gives rise to the “classical” receptive field properties –tuning properties exhibited in very first spikes Orientation tuning in V1 Optic flow tuning in MST –think of cortical neurons as “detectors” only during feed- forward sweep

18. After the Forward Sweep By 150 ms, virtually every visual brain area has responded to the onset of a visual stimulus But visual cortex neurons continue to fire for hundreds of milliseconds!

19. After the Forward Sweep By 150 ms, virtually every visual brain area has responded to the onset of a visual stimulus But visual cortex neurons continue to fire for hundreds of milliseconds! What are they doing?

20. After the Forward Sweep By 150 ms, virtually every visual brain area has responded to the onset of a visual stimulus But visual cortex neurons continue to fire for hundreds of milliseconds! What are they doing? with sufficient time (a few tens of ms) neurons begin to reflect aspects of cognition other than “detection”

21. Extra-RF Influences One thing they seem to be doing is helping each other figure out what aspects of the entire scene are contained within a given receptive field –That is, the responses of visual neurons begin to change to reflect global rather than local features of the scene –recurrent signals sent via feedback projections are thought to mediate these later properties

22. Extra-RF Influences Note that these are responses to the same stimulus!

23. Extra-RF Influences consider texture-defined boundaries –classical RF tuning properties do not allow neuron to know if RF contains figure or background –At progressively later latencies, the neuron responds differently depending on whether it is encoding boundaries, surfaces, the background, etc.

24. Extra-RF Influences Consider this analogy: –Imagine when each fan puts up a card he or she is told to shake it – so that the entire scene is full of shaking cards –After some delay, the fans holding up the red cards are told to keep shaking but the fans holding white cards are told to stop…the words will be enhanced –But the fans can’t each figure that out on their own because they don’t actually know the color of the card they are holding

25. Extra-RF Influences How do these data contradict the notion of a “classical” receptive field?

26. Extra-RF Influences How do these data contradict the notion of a “classical” receptive field? Remember that for a classical receptive field (i.e. feature detector): –If the neuron’s preferred stimulus is present in the receptive field, the neuron should fire a stereotypical burst of APs –If the neuron is firing a burst of APs, its preferred stimulus must be present in the receptive field

27. Extra-RF Influences How do these data contradict the notion of a “classical” receptive field? Remember that for a classical receptive field (i.e. feature detector): –If the neuron’s preferred stimulus is present in the receptive field, the neuron should fire a stereotypical burst of APs –If the neuron is firing a burst of APs, its preferred stimulus must be present in the receptive field

28. Recurrent Signals in Object Perception Can a neuron represent whether or not its receptive field is on part of an attended object? What if attention is initially directed to a different part of the object?

29. Recurrent Signals in Object Perception Can a neuron represent whether or not its receptive field is on part of an attended object? What if attention is initially directed to a different part of the object? Yes, but not during the feed-forward sweep

30. Recurrent Signals in Object Perception curve tracing –monkey indicates whether a particular segment is on a particular curve –requires attention to scan the curve and “select” all segments that belong together –that is: make a representation of the entire curve –takes time

31. Recurrent Signals in Object Perception curve tracing –neuron begins to respond differently at about 200 ms –enhanced firing rate if neuron is on the attended curve

32. Feedback Signals and the binding problem What is the binding problem?

33. Feedback Signals and the binding problem What is the binding problem? curve tracing and the binding problem: –if all neurons with RFs over the attended curve spike faster/at a specific frequency/in synchrony, this might be the binding signal

34. Feedback Signals and the binding problem So what’s the connection between Attention and Recurrent Signals?

35. Feedback Signals and Attention One theory is that attention (attentive processing) entails the establishing of recurrent “loops” This explains why attentive processing takes time - feed-forward sweep is insufficient

36. Feedback Signals and Attention Instruction cues (for example in the Posner Cue- Target paradigm) may cause feedback signal prior to stimulus onset (thus prior to feed-forward sweep) think of this as pre-setting the system for the upcoming stimulus What does this accomplish?

37. Feedback Signals and Attention What does this accomplish? Preface to attention: Two ways to think about attention –Attention improves perception, acts as a gateway to memory and consciousness –Attention is a mechanism that routes information through the brain It is the brain actively reconfiguring itself by changing the way signals propagate through networks It is a form of very fast, very transient plasticity

38. Feedback Signals and Attention Put another way: –It may strike you as remarkable that a single visual stimulus should “activate” so many brain areas so rapidly –In fact it should be puzzling that a visual input doesn’t create a runaway “chain reaction” The brain is massively interconnected Why shouldn’t every neuron respond to a visual stimulus

39. Feedback Signals and Attention We’ll consider the role of feedback signals in attention in more detail as we discuss the neuroscience of attention