Computational Vision CSCI 363, Fall 2012 Lecture 14 Spatial Frequency in Striate Cortex

Another viewpoint: V1 cells are spatial frequency filters DeValois (and others) proposed that V1 cells are tuned to spatial frequency. V1 simple cells can be modeled as Gabor filters. A Gabor filter is the product of a sinewave and a gaussian. sin(10x) G(x) sin(10x)G(x)

Fourier Transform of a Gabor Filter The Fourier Transform of a Gabor filter is a localized set of spatial frequencies. Gabor filters are band-pass filters. They are tuned to spatial frequency. If Striate Cortex cells are like Gabor filters, then they are also acting as band-pass filters. Fourier Transform

2D Gabor filter

Simple Cell Receptive Field vs. Gabor Function Solid line: Simple Cell Receptive Field. Dashed line: Best fitting Gabor function. From: DeValois and DeValois, "Spatial Vision", 1988.

Contrast Sensitivity of V1 Cells Distribution of tuning bandwidth CSF for individual V1 cells

Spatial Profile vs. CSF The spatial profile of the simple cell receptive field is predicted by taking the inverse Fourier transform of the contrast sensitivity function for that cell.

Spatial Frequency Columns As with orientation and ocular dominance, spatial frequency shows columnar organization in the cortex.

Simple vs. Complex Cells The response of simple cells to drifting gratings shows a big oscillation over time. The complex cell response does not oscillate much.

Classifying simple and complex cells Simple cells have a larger AC (Alternating current) response. Complex cells have a larger DC (Direct current) response. The ratio of AC/DC allows classification. AC/DC >1 => simple cells. AC/DC < 1 => complex cells

Temporal Frequency Temporal frequency is the frequency of oscillation of light intensity over time. For a drifting sinewave grating, the luminance at a single point in space oscillates over time. http://web.mit.edu/~jgolomb/www/drifting.gif (Prof. Julie Golomb, MIT) Temporal frequency is measured in cycles/sec = Hertz (Hz) Temp. Freq. (cycles/sec) = Speed (deg/sec) x Spatial Freq (cycles/deg) Faster drift => Higher temporal frequency Higher spatial frequency => Higher temporal frequency

Speed vs. Temporal Frequency To test whether V1 cells are tuned to speed or temporal frequency, Tolhurst and Movshon (and others) examined speed tuning for sinewave gratings with different spatial frequencies. Example: Test spatial frequencies of 1.0, 2.0 and 3.0 cycles/deg. If neurons are tuned for speed, the tuning curves will peak at the same speed , independent of spatial frequency. (For example, if the neuron prefers 5 deg/sec, all the graphs will have a peak there). If neurons are tuned for temporal frequency (e.g. 10 cycles/sec), then the peaks for speed tuning will depend on the spatial frequency (The peaks will be at 10, 5 and 3.33 deg/sec). Cells in striate cortex appear to be tuned to temporal frequency. (Cells at higher levels may be tuned to speed).

V1 cells and Temporal Frequency Response of neuron tuned for speed. MT cells match this pattern. Response of neuron tuned for temporal frequency. V1 cells match this pattern Simoncelli and Heeger, Nature Neuroscience, 2001

Linear Systems Linear functions: F(x1 + x2) = F(x1) + F(x2) F(ax) = aF(x) Linear systems are nice to work with because you can predict (or compute) the responses of the system relatively easily. For example, if you double the input, the output doubles. Fourier Transforms are linear operations. (The Fourier transform of the sum of two images is the sum of the Fourier transforms of each image). Gabor filters are linear filters. Neurons are not linear.

Threshold and Saturation Threshold non-linearity: Neurons do not respond until the input reaches a minimum level (threshold). Saturation non-linearity: Neurons have a maximum firing rate. The response saturates after they reach this maximum. Threshold Saturation Response Linear response region Input strength

Phase and Half-wave Rectification Phase non-linearity: Complex cells are insensitive to the phase (position) of a grating within the receptive field. Complex cells do not sum inputs within the receptive field. Half-wave Rectification: Cortical cells have a low spontaneous firing rate. There cannot be as large a negative response as a positive response. The bottom half of the waveform is clipped off. This can be alleviated with pairs of matched cells that are 180 deg out of phase with one another. The difference in responses acts like a linear response. Response time

Lateral Inhibition There is evidence that a spatial frequency channel is inhibited by other channels tuned to nearby frequencies. (Also true for orientation tuning). This is accomplished by lateral inhibitory connections within the cortex, known as lateral inhibition. This can cause interesting effects, such as repulsion of perceived orientation when 2 lines of similar orientation are shown close together. If you adapt 1 spatial frequency, there is an increased sensitivity at other nearby frequencies. Inhibitory interactions can help to make tuning curves narrower.