1. Computer Vision Lecture 2: Vision, Attention, and Eye Movements
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

2. Computer Vision Lecture 2: Vision, Attention, and Eye Movements
Stimuli in receptive field of neuron
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

3. Computer Vision Lecture 2: Vision, Attention, and Eye Movements
Cat V1 (striate cortex)
Orientation preference map
Ocular dominance map
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

4. Computer Vision Lecture 2: Vision, Attention, and Eye Movements
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

5. Structure of NNs (and some ANNs)
In biological systems, neurons of similar functionality are usually organized in separate areas (or layers).
Often, there is a hierarchy of interconnected layers with the lowest layer receiving sensory input and neurons in higher layers computing more complex functions.
For example, neurons in macaque visual cortex have been identified that are activated only when there is a face (monkey, human, or drawing) in the macaque’s visual field.
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

6. Computer Vision Lecture 2: Vision, Attention, and Eye Movements
“Data Flow Diagram”
of Visual Areas in
Macaque Brain
Blue: motion perception pathway
Green: object recognition pathway
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

7. Receptive Fields in Hierarchical Neural Networks
neuron A
Complexity tells us to use pyramids - but new problem...
Routing is one of those problems that was little considered before Anderson & Van Essen pointed it out (Koch & Ullman did say selected features routed to high level, but not how). They used shifts in space (a giant multiplexor) controlled by outside signals to route a signal from anywhere int he input field to the appropriate portions of the output layer. Claimed that stereo,motion can also be handled analogously. Never implemented or tested the idea; in Olshausen’s thesis. connectivity was prohibitive. ‘Patchy’ connectivity solved this.
Burt used a Laplacian pyramid: each level is determined as the difference of the previous two levels of the Guassian pyramid. Fovea imposed on this with 1/theta acuity solves routing. Only attended signal reaches output (rest is discarded at each level)
Key to my routing is observation that stimulus feeds a forward cone and that any output ayer unit is root of a sub-pyramid. Potential for cross-talk is large
Single attentional fixation at a time - can’t route two without interference!
receptive field of A
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

8. Receptive Fields in Hierarchical Neural Networks
neuron A
in top layer
Complexity tells us to use pyramids - but new problem...
Routing is one of those problems that was little considered before Anderson & Van Essen pointed it out (Koch & Ullman did say selected features routed to high level, but not how). They used shifts in space (a giant multiplexor) controlled by outside signals to route a signal from anywhere int he input field to the appropriate portions of the output layer. Claimed that stereo,motion can also be handled analogously. Never implemented or tested the idea; in Olshausen’s thesis. connectivity was prohibitive. ‘Patchy’ connectivity solved this.
Burt used a Laplacian pyramid: each level is determined as the difference of the previous two levels of the Guassian pyramid. Fovea imposed on this with 1/theta acuity solves routing. Only attended signal reaches output (rest is discarded at each level)
Key to my routing is observation that stimulus feeds a forward cone and that any output ayer unit is root of a sub-pyramid. Potential for cross-talk is large
Single attentional fixation at a time - can’t route two without interference!
receptive field of A in input layer
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

9. Computer Vision Lecture 2: Vision, Attention, and Eye Movements
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

10. Face aftereffect – Thanks to Arash Afraz for the slides!
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

11. Computer Vision Lecture 2: Vision, Attention, and Eye Movements
Visual Illusions
Visual Illusions demonstrate how we perceive an “interpreted version” of the incoming light pattern rather that the exact pattern itself.
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

12. Computer Vision Lecture 2: Vision, Attention, and Eye Movements
Visual Illusions
He we see that the squares A and B from the previous image actually have the same luminance (but in their visual context are interpreted differently).
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

13. Computer Vision Lecture 2: Vision, Attention, and Eye Movements
Visual Attention
Visual attention is the selective allocation of visual processing resources.
For example, we can focus our attention on a particular object of interest in the visual field.
Visual processing of that object is enhanced while being rather shallow for other objects.
Also, we can respond more quickly and accurately to changes in an attended region.
This prioritization is necessary due to our limited processing resources.
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

14. Computer Vision Lecture 2: Vision, Attention, and Eye Movements
Visual Attention
The attentional cueing task introduced by Michael Posner gives insight into the dynamics of visual attention.
Subjects are instructed to fixate on the central cross.
One of two boxes (left or right) flashes to capture the subject’s attention (an automatic, involuntary response).
After some a short delay (stimulus onset asynchrony - SOA) an asterisk appears in one of the boxes.
The subject has to report as quickly as possible in which box the asterisk appeared.
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

15. The Posner Attention Taskx
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

16. The Posner Attention Taskx
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

17. The Posner Attention Taskx
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

18. The Posner Attention Task* x
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

19. The Posner Attention Taskx
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

20. The Posner Attention Task
For short SOAs (< 200 ms), subjects respond faster if flash and asterisk appear on the same side.
Cueing of attention to relevant location allows faster response.
For longer SOAs, subjects respond more slowly if flash and asterisk appear on the same side.
Inhibition-of-Return mechanism makes attention less likely to remain on the side of the flash until the asterisk appears.
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

21. Computer Vision Lecture 2: Vision, Attention, and Eye Movements
Eye Muscles
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

22. Computer Vision Lecture 2: Vision, Attention, and Eye Movements
Types of Eye Movement
Fixations:
The eye is almost motionless, for example, while reading a single, short word.
The information from the scene is almost entirely acquired during fixation.
Duration varies from ms, typically between ms.
Typical fixation frequency is about 3 Hz.
Fixations are interspersed with saccades.
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

23. Computer Vision Lecture 2: Vision, Attention, and Eye Movements
Types of Eye Movement
Saccades:
Quick “jumps” that connect fixations
Duration is typically between 30 and 120 ms
Very fast (up to 700 degrees/second)
Saccades are ballistic, i.e., the target of a saccade cannot be changed during the movement.
Vision is suppressed during saccades to allow stable perception of surroundings.
Saccades are used to move the fovea to the next object/region of interest.
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

24. Computer Vision Lecture 2: Vision, Attention, and Eye Movements
Types of Eye Movement
Smooth Pursuit Eye Movements:
Smooth movement of the eyes for visually tracking a moving object
Cannot be performed in static scenes (fixation/saccade behavior instead)
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

25. Why Eye-Movement Research?
About eye movements and visual attention:
Usually, saccades follow shifts of attention to provide high acuity at the attended position.
It is possible to look at an object without paying attention to it (staring).
It is possible to shift attention without eye movement (covert shifts of attention).
It is impossible to perform a saccade while not shifting attention.
During specific, natural tasks it is reasonable to assume that saccades follow shifts of attention.
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

26. Why Eye-Movement Research?
The investigation of visual attention, in turn, is at the core of cognitive science.
Studying visual attention yields insight into general attentional mechanisms.
It can provide information on a person’s stream of conscious and unconscious processing while solving a task.
Attention is closely linked to the concept of consciousness.
Attentional mechanisms could improve artificial vision systems.
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

27. Computer Vision Lecture 2: Vision, Attention, and Eye Movements
Eye-Movement Studies
Eye movements while watching a girl’s face (early study by Yarbus, 1967)
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

28. Computer Vision Lecture 2: Vision, Attention, and Eye Movements
Eye-Movement Studies
Eye movements as
indicators of cognitive
processes (Yarbus):
trace 1: examine at will
trace 2: estimate wealth
trace 3: estimate ages
trace 4: guess previous activity
trace 5: remember clothing
trace 6: remember position
trace 7: time since last visit
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

29. Computer Vision Lecture 2: Vision, Attention, and Eye Movements
Eye-Movement Studies
Visual scan paths on instruments/dashboards – studies for the improvement of human-computer interfaces
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

30. Eye-Movement Studies
Gaze trajectory measurement for the optimization of web page layout
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

31. Eye-Movement Studies
Improving advertisements with eye-movement studies
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

32. Selectivity in Complex Scenes
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements

33. Computer Vision Lecture 2: Vision, Attention, and Eye Movements
Face Recognition
Gaze-contingent window deteriorates face recognition, allows to identify relevant visual information.
January 25, 2018
Computer Vision Lecture 2: Vision, Attention, and Eye Movements