Gabor Filter: A model of visual processing in primary visual cortex (V1) Presented by: CHEN Wei (Rosary) Supervisor: Dr. Richard So

Agenda Visual Physiological Mechanism Primary Visual Cortex (V1) Properties Gabor Filter Model Implications Q & A

Anatomy of the Early Visual Pathways light retina optic nerve LGN optic radiation primary visual cortex

Visual Pathway Light fall into the eye, form two images on the left and right retina Each retina transforms the incoming light intensity distribution into spike patterns, transmitted by optic nerves to the Central nervous system (CNS) CNS target: Lateral Geniculate Nuclei(LGN) of the thalamus Proceeding from the LGN, optic radiation contracts the primary visual cortex (area 17 or V1, or Striate Cortex)

Primary Visual Cortex (V1) A thin sheet of neurons The largest cortical area in primates, containing some 200 million cells Complex visual processing occurs here Well known area in brain

What are cells in V1 doing? Hubel and Wiesel’s outstanding research and other studies revealed: cells in the primary visual cortex have receptive fields which are restricted to small regions of space and highly structured. They are working as linear filters to decompose the visual image These cells are highly orientation and spatial frequency selectively

Spatial Frequency Theory Atomistic assumption: the representation of any image, no matter how complex, is an assemblage of many primitive spatial “atoms” spatially extended patterns called sinusoidal gratings

Sine Grating Luminance profile is described as sine wave Important parameters: Amplitude contrast Spatial frequency (cycle/degree) Orientation phase

Image decompose Any 2D image can be decomposed into a unique set of sine gratings According to Fourier’s Theorem, a complex pattern (like a face) is just the sum of a particular set of sine gratings

A Cell is a filter Sachs, Nachmias (1971), H.R.Wilson (1979,1993) etc. Each cell is trying to decompose the incoming information to particular component Studies show the stimulus is processed by mechanisms selective for both spatial frequency and orientation. The response of each mechanism is produced by linear filtering by its receptive field

Spatial frequency selective V1 cells respond to narrow ranges of stimulus spatial frequency

Orientation Selective Similarly, V1 cells respond to narrow ranges of stimulus Orientation Preferred orientation: only excitation=big response Orthogonal orientation: excitation and inhibition cancel out= no response

V1 functional architecture Receptive field: linear, weighted sum of stimulus intensities. This model is attractive because it allows for a complete description. Measure the linear weighting function for a given cell, you can predict that cell’s response to any visual stimulus

In cats, the spatial frequency dimension is hypothesized to be orthogonal to the orientation dimension in a Cartesian structure, whereas in monkeys it is hypothesized to be radially organized in a polar structure How do we model both spatial frequency and orientation Selectivity Simultaneously??

Gabor Filter Model A mathematical model used to describe receptive field properties of the cells in V1 The receptive field structure is called a Gabor filter (or wavelet) in constructed by multiplying a global sinusoidal grating by a bell-shaped Gaussian envelope. Describe the 2D information

Elliptic Gaussian A 2D elliptic Gaussian centered on the origin of a Cartesian coordinate system: Where, is variance in the x direction is variance in the y direction

Elliptic Gaussian cont. The elliptic Gaussian can be centered at any desired spatial location traslating offset x 0 and y 0 and aligned in an arbitrary orientation by a rotation of angle A Where,

Sine Plane Wave A 2D sinusoidal plane wave can be written Where, is spatial frequencies in the x direction is spatial frequencies in the y direction

Sine Wave Plane cont. Since only the real signals (cosine term) can be observed in a experiment and we can choose some other origin for the modulation term,

Observable form of the 2D Gabor filter in the space domain The 2D Gabor filter can now be written as the product of an elliptic Gaussian and a sinusoidal plane wave K --- scale factor (amplitude)

sample output of Gabor filters with different scales and different orientations 0 0, scale , scale , scale , scale , scale 5

The evaluation of Gabor filter Jones and Palmer (1987) concluded that the Gabor function provides a useful and reasonably accurate description of cells in cat’s V1 In Jones & Palmer´s experiment, simple cells responses were measured with a micro electrode. refer to Judson and Palmer, 1987, The two-Dimensional spatial structure of simple receptive fields in cat striate cortex, Journal of Neurophysiology

An example of the resulting responses left: experimental data in the space domain middle: adapted Gabor filter right: difference of the adapted filter values to the experimental data

Conclusion of the evaluation The differences between the data and the fits were statistically indistinguishable from random error Simple receptive fields in cat primary visual cortex are linear filters having the function form of 2D Gabor filters

An Example: Output after Gabor Filter

Implications Early visual processing (V1) is commonly modeled with a population of visual filters selective for different orientations and spatial frequencies. Besides V1 model,we are studying more models related to computational modeling to gain insight into biological brain visual function

Implications Thus, we study biologically-plausible brain models, and compare the predictions resulting from model simulations to empirical measurements from living systems. The target: We are trying to build a quantitative computational model to predict the severity of visually-induced motion sickness

Thank You! Your comments and suggestions are appreciated

Linear Filters It is a computation which takes one sequence of numbers (the input signal) and produces a new sequence of numbers (the filtered output signal). A filter is linear means simply that Scaling: the amplitude of the output is proportional to the amplitude of the input Superposition:when two signals are added together and fed to the filter, the filter output is the same as if one had put each signal through the filter separately and then added the outputs

Visual cortical pathways after V1 V1 forms an “internal representation” Neurons in higher cortical areas make use of the internal representation to infer what is out there in the world. 30 or more secondary visual areas after V1 in the occipital lobe and in parts of the parietal and temporal lobes, for example, MT: visual motion processing area Handle where/what, motion processing involved in motion analysis

Data Fit algorithm In the space domain, it was found 2D Gabor filters that fit the 2D spatial response profile of each simple cell in the least squared error sense (with a simplex algorithm), and it was shown that the residual error is devoid of spatial structure and statistically indistinguishable from random error.

Primary Visual Cortex Hubel and Wiesel (1959) revealed: Simple cells: their response to complex stimuli can be predicted from their response to individual spots of light. A simple cell’s receptive field can therefore be mapped just by determining it’s response to a small spot of light at each position on the retina Complex cells:integrating the response of many simple cells Hypercomplex cells:end-stopped simple or complex cells