Computational Vision CSCI 363, Fall 2012 Lecture 22 Motion III

Gradient Models The gradient models use the "Contrast Brightness Assumption". In 1 spatial dimension, this states: t0 t0 + dt I I x0 x x0 + dx x

The Gradient Constraint Equation The Gradient Constraint Equation in 1 dimension: The Gradient Constraint Equation in 2 dimensions:

Motion Energy Models An alternative way to think about 2D motion detection involves using spatio-temporal frequency filters. This type of model relies on filters that are combined to detect a certain range of spatial and temporal frequencies. Various people have developed versions of these models: van Santen & Sperling Watson & Ahumada Adelson & Bergen

Motion as orientation In x-t space, motion is an oriented line. The slant depends on speed. In x-y-t space, motion becomes an oriented slab within a volume. x-t x-y-t

Orientation Detectors in Space-Time Oriented Spatio-temporal filters: Filters oriented in space time can detect a moving stimulus. The orientation of the filter relates to its preferred speed of motion. These filters can detect sampled motion as well.

Separable Spatio-temporal filters A Spatio-temporal filter can be created as the product between a spatial filter and a temporal filter. Spatial impulse response = HS(x) Temporal impulse response = HT(t) Spatio-temporal impulse response: HST(x, t) = HS(x)HT(t)

Response to a Moving Edge There is little response at t1 and t3. There is largest response at t2 during the edge motion

Oriented spatio-temporal filters The previous filter was not selective for direction of motion. We can develop an oriented filter that is selective for direction, by creating a spatio-temporal Gabor filter: - + Filter selective for leftward motion

Response of oriented filter Non-oriented Oriented Moving edge stimulus Filter Response

Problems with Gabor filter The Gabor filter by itself results in several problems: It is phase sensitive: It depends on a particular alignment of the pattern with the filter at a given time. (The response to a drifting sinewave is an oscillation). 2) The sign of the response depends on the stimulus contrast (e.g. white on black gives opposite response to black on white).

Solution: Motion Energy Motion energy filters are constructed with 2 gabor filters, one of which uses a sine and the other uses a cosine (a "quadrature pair"). If you square the outputs of the gabors and sum, the result is motion energy.

Motion Energy Responses With motion energy filters: The response is always positive. The response is the same for a black-white edge as for a white-black edge. The response to motion is independent of contrast. The response is constant for a drifting sinewave.

Motion-Opponency Psychophysical results suggest that neurons in the brain use a motion-opponent processing (e.g. left - right). Evidence: We cannot see both left and right motion at the same time. If you superimpose a leftward moving sinewave pattern on a rightward moving sinewave pattern, you don't see motion, you see flicker. 2) Motion after-effect: If you adapt to rightward motion, and then look away at static image, you see leftward motion (The waterfall illusion). Demo: http://www.michaelbach.de/ot/mot_adapt/index.html

Construction of Motion Opponent energy filters To construct a motion-opponent energy filter, simply subtract the response of a filter tuned to left motion from that of a filter tuned to right motion (or vice versa).

Velocity Extraction Problem: Velocity information confounded with contrast. E.g. A weak signal could mean low contrast or low velocity. Solution: Compare the output of different motion channels. (E.g. Left, right and static channels. Change in contrast => Ratio between channels stays the same Change in velocity => Ratio between channels changes.

Reverse Phi Energy White energy => Move pattern in If a pattern of white and black lines is moved rightward in small steps, people see rightward motion. If the contrast is reversed with each step (white becomes black and vice versa), people see leftward motion. (The Reverse Phi Effect) Demo: http://psy2.ucsd.edu/~sanstis/Stuart_Anstis/Reverse_phi.html Energy White energy => rightward motion. Move pattern in steps. Reverse Phi: Move pattern in steps while reversing contrast Dark energy => leftward motion.

Fluted Square Wave A square wave that is shifted to the right in 90 deg steps, appears to move right. 90 deg step to right A square wave with the fundamental frequency component removed is a fluted square wave. The highest amplitude component is 3f. When the fluted square wave is shifted to the right in 90 deg steps, it appears to move left!

Fluted Square Wave Why? When a square wave that is shifted to the right in 90 deg steps, its fundamental frequency moves right in 90 deg steps. For a fluted square wave, the highest amplitude component is 3f. When the square wave (frequency f) moves 90 deg to the right, the 3f component is being shifted 270 deg to the right, which appears as 90 deg to the left.

Energy Response to a fluted square wave x Energy White = Right Square wave t Fluted Square wave Black = Left

Moving Plaid Demo Demo of a moving plaid grating: = + Demo: http://www.viperlib.com (Search on "plaid", choose coherence.mp4)