1 Computational Vision CSCI 363, Fall 2012 Lecture 32 Biological Heading, Color

2 How does the brain process heading? It is not known how the brain computes observer heading, but there are numerous models and hypotheses. One of the simplest ideas is based on template models: Neurons in the brain are tuned to patterns of velocity input that would result from certain observer motions. Support for this idea: Tanaka, Saito and others found cells in the dorsal part of the Medial Superior Temporal area (MSTd) that respond well to radial, circular or planar motion patterns. Since then, people have assumed that MSTd is involved in heading computation.

3 Visual Pathway

4 Types of Responses in MSTd (from Duffy & Wurtz, 1991)

5 Combinations of Patterns Duffy and Wurtz (1991) tested cell responses to planar, circular and radial patterns. They found some cells responded only to one type of pattern (e.g. only to circular). Others responded to two or three types of patterns (e.g. both planar and circular). Single ComponentDouble ComponentTriple Component RadialPlano-RadialPlano-Circulo-Radial CircularPlano-Circular Planar They did not suggest a model of how these might be involved in heading detection. They also showed there is not a simple way that MST receptive fields are made from inputs from MT cell receptive fields.

6 Spiral Patterns Graziano et al. (1994) showed that MSTd cells respond to spiral patterns of motion:

7 Does MST compute heading? Prediction: If MST is involved in heading computation, one would expect to find cells tuned to a particular position for the center of expansion. Duffy and Wurtz (1995) tested this prediction.

8 Do MSTd cells use eye- movement information? Psychophysical experiments showed that humans can make use of eye movement information to compute heading. Some MSTd cells have responses that are modulated by eye movements. Do eye movements affect the responses of MSTd cells to compensate for rotation? This was tested in an experiment by Bradley et al (1996). They recorded from MSTd cells while showing flow fields that consisted of an expansion plus a rotation. The rotation was generated by real or simulated eye movements.

9 Real eye movement condition

10 Simulated eye movement condition

11 Results No eye movement Eye movement in preferred direction. Eye movement in anti-preferred direction. This cell seems to take into account eye movements. The effect was not consistent among all cells tested.

12 Recall Dorsal and Ventral Streams

13 Wavelengths and Color 1. Light is composed of photons with different wavelengths. 2. Humans are sensitive to a small range of wavelengths: nm.

14 Light Spectra White light is a mixture of all colors: A variety of wavelengths of light come from different light sources. These are reflected in the spectra of the light. The light reflected off surfaces can be characterized by spectra as well.

15 Color Perception is not equivalent to wavelength When light hits a surface, some is reflected and some is absorbed and some may be transmitted, depending on the surface properties. Some of the reflected light may reach the eye and is focused on the retina. Neural processing of the light that hits the retina determines our perception of color. Perception of color is not an inherent object property. It depends on neural processing. Different people may perceive the same wavelengths (or spectra) differently (e.g colorblind people).

16 Color Space Human color perception is defined in terms of 3D color space. The 3 dimensions of this space are Hue, Saturation and Brightness. 2 sides of color space (or spindle). brightness goes from bottom (dark) to top (light).

17 Hue and Saturation A slice through the color spindle shows a color circle of a given brightness. Hue is the direction from the center of the circle. It varies as you move around the circumference of the circle. Saturation is the distance from the center of the circle. The most vibrant colors (high saturation) are at the outside of the circle. The grayest colors (low saturation) are at the inside of the circle.

18 Another view of Hue, Saturation and Brightness In a simple case, if the spectrum forms a Gausian distribution, then: Hue is given by the mean of the Gaussian. Saturation is given by the variance (width) of the Gaussian. Brightness is given by the area under the Gaussian curve.

19 Trichromatic Color Theory Observation: You can match any color in color space with a linear combination of 3 appropriately chosen (no co-linear) colors. Color Matching "Subtract" a color by adding it into the test color.

20 Three types of cone photoreceptors The retina has three types of cone photoreceptors. Each has a different pigment molecule that absorbs best at a given wavelength. S (Short wavelength), "Blue"=>440 nm M (Middle), "Green" => 530 nm L (Long), "Red" => 560 nm A single wavelength stimulus will stimulate each type of cone by some amount. A mixed color match is a set of stimuli that stimulate the cones by the same amounts as the single wavelength.

21 1 cone cannot detect color 1 cone type: Cannot tell change in brightness from change in Hue. 2 cone types: Can distinguish some colors. 3 cone types: Can distinguish more colors.

22 Color perception requires comparison of responses A given wavelength of light will cause a set of responses in the three receptor types. If the intensity changes, the ratio of responses between the photoreceptors will stay the same (but all will increase or decrease). If the wavelength (or spectrum) changes, the ratio of responses between the photoreceptors will change. Next Lecture: Why are certain colors complementary? (E.g. Red/Green or Blue/Yellow).