1. Computational Vision CSCI 363, Fall 2012 Lecture 22 Motion III

2. Gradient Models
The gradient models use the "Contrast Brightness Assumption".
In 1 spatial dimension, this states: t0
t0 + dt I I x0 x
x0 + dx x

3. The Gradient Constraint Equation
The Gradient Constraint Equation in 1 dimension:
The Gradient Constraint Equation in 2 dimensions:

4. Motion Energy Models
An alternative way to think about 2D motion detection involves using spatio-temporal frequency filters.
This type of model relies on filters that are combined to detect a certain range of spatial and temporal frequencies.
Various people have developed versions of these models:
van Santen & Sperling
Watson & Ahumada
Adelson & Bergen

5. Motion as orientation
In x-t space, motion is an oriented line. The slant depends on speed.
In x-y-t space, motion becomes an oriented slab within a volume.
x-t
x-y-t

6. Orientation Detectors in Space-Time
Oriented Spatio-temporal filters:
Filters oriented in space time can detect a moving stimulus.
The orientation of the filter relates to its preferred speed of motion.
These filters can detect sampled motion as well.

7. Separable Spatio-temporal filters
A Spatio-temporal filter can be created as the product between a spatial filter and a temporal filter.
Spatial impulse response = HS(x)
Temporal impulse response = HT(t)
Spatio-temporal impulse response: HST(x, t) = HS(x)HT(t)

8. Response to a Moving Edge
There is little response at t1 and t3.
There is largest response at t2 during the edge motion

9. Oriented spatio-temporal filters
The previous filter was not selective for direction of motion.
We can develop an oriented filter that is selective for direction, by creating a spatio-temporal Gabor filter: - +
Filter selective for leftward motion

10. Response of oriented filter
Non-oriented
Oriented
Moving edge
stimulus
Filter
Response

11. Problems with Gabor filter
The Gabor filter by itself results in several problems:
It is phase sensitive: It depends on a particular alignment of the pattern with the filter at a given time. (The response to a drifting sinewave is an oscillation).
2) The sign of the response depends on the stimulus contrast (e.g. white on black gives opposite response to black on white).

12. Solution: Motion Energy
Motion energy filters are constructed with 2 gabor filters, one of which uses a sine and the other uses a cosine (a "quadrature pair").
If you square the outputs of the gabors and sum, the result is motion energy.

13. Motion Energy Responses
With motion energy filters:
The response is always positive.
The response is the same for a black-white edge as for a white-black edge.
The response to motion is independent of contrast.
The response is constant for a drifting sinewave.

14. Motion-Opponency
Psychophysical results suggest that neurons in the brain use a motion-opponent processing (e.g. left - right).
Evidence:
We cannot see both left and right motion at the same time.
If you superimpose a leftward moving sinewave pattern on a rightward moving sinewave pattern, you don't see motion, you see flicker.
2) Motion after-effect: If you adapt to rightward motion, and then look away at static image, you see leftward motion (The waterfall illusion).
Demo:

15. Construction of Motion Opponent energy filters
To construct a motion-opponent energy filter, simply subtract the response of a filter tuned to left motion from that of a filter tuned to right motion (or vice versa).

16. Velocity Extraction
Problem: Velocity information confounded with contrast.
E.g. A weak signal could mean low contrast or low velocity.
Solution: Compare the output of different motion channels. (E.g. Left, right and static channels.
Change in contrast => Ratio between channels stays the same
Change in velocity => Ratio between channels changes.

17. Reverse Phi Energy White energy => Move pattern in
If a pattern of white and black lines is moved rightward in small steps, people see rightward motion.
If the contrast is reversed with each step (white becomes black and vice versa), people see leftward motion. (The Reverse Phi Effect)
Demo:
Energy
White energy =>
rightward motion.
Move pattern in
steps.
Reverse Phi:
Move pattern in
steps while reversing contrast
Dark energy =>
leftward motion.

18. Fluted Square Wave
A square wave that is shifted to the right in 90 deg steps, appears to move right.
90 deg step to right
A square wave with the fundamental frequency component removed is a fluted square wave. The highest amplitude component is 3f.
When the fluted square wave is shifted to the right in 90 deg steps, it appears to move left!

19. Fluted Square Wave
Why? When a square wave that is shifted to the right in 90 deg steps, its fundamental frequency moves right in 90 deg steps.
For a fluted square wave, the highest amplitude component is 3f.
When the square wave (frequency f) moves 90 deg to the right, the 3f component is being shifted 270 deg to the right, which appears as 90 deg to the left.

20. Energy Response to a fluted square wavex
Energy
White = Right
Square wave t
Fluted
Square wave
Black = Left

21. Moving Plaid Demo Demo of a moving plaid grating: = + Demo:
(Search on "plaid", choose coherence.mp4)