1. 1 Computational Vision CSCI 363, Fall 2012 Lecture 21 Motion II

2. 2 Gradient Models The gradient models use the "Contrast Brightness Assumption". In 1 spatial dimension, this states: I xx0x0 t0t0 I x x 0 +  x t 0 +  t

3. 3 The Gradient Constraint Equation The Gradient Constraint Equation in 1 dimension: The Gradient Constraint Equation in 2 dimensions:

4. 4 The Aperture Problem The gradient constraint equation for a 2D image is 1 equation with 2 unknowns (u and v). To solve for u and v, we must make measurements of I x, I y, and I t at 2 locations where they are not all identical. If our view is limited to an edge seen through an aperture, we cannot solve for both u and v independently. We can only find the component of motion perpendicular to the edge. Aperture Edge Perpendicular velocity component

5. 5 The Aperture Problem is Fundamental The aperture problem is a fundamental problem when one is trying to measure image velocity using local detectors. This is true in biological vision (neurons have local receptive fields). This is also true in machine vision (intensity is detected locally by photodetectors).

6. 6 Solving the Aperture problem 1. Assume pure translation of the object. The true velocity may lie anywhere along this "constraint" line. 2. Make two separate local measurements. v1v1 v2v2 Replot in velocity space v2v2 v1v1 vxvx vyvy The true velocity is at the intersection of the constraint lines.

7. 7 The Smoothness Constraint The previous solution to the aperture problem requires a rigid object that is not rotating. If the object is rotating or changing shape (deforming), we need another constraint to solve for velocity. The "smoothness constraint" states that the velocity along a boundary (or within a 2D area of the image) varies smoothly. Note: This is violated across the boundary of a moving object.

8. 8 Measuring Velocity along a Contour SS v C vyvy vxvx Total variation over the curve is: To impose the smoothness constraint, we find the velocity field that minimizes the above integral.

9. 9 Do Humans use a smoothness constraint? The model incorporating the smoothness constraint finds the correct result for: 1) Pure translation of an object 2) General motion of a rigid object with straight edges. Nakayama and Silverman developed a stimulus that shows that humans integrate along contours and over small 2D area: Oscillating contour Looks non-rigid Add line-breaks Looks rigid Add lines Looks rigid

10. 10 Motion Illusions The model fails for several cases where humans also do not see the correct velocity: 1) Rotating spirals (look like they are expanding) http://www.michaelbach.de/ot/mot_adaptSpiral/index.html 2) The Barberpole illusion (looks like it is moving up) http://www.123opticalillusions.com/pages/barber_pole.php 3) "Wobbling" ellipses. The model computes velocities along the contours that are consistent with human perception. Note: A few experiments have shown this is not exactly true all the time.

11. 11 Rotating spirals True velocity vectors Initial measurements Smoothest velocity field from initial measurements

12. 12 Barberpole illusion True velocity vectors Smoothest velocity field

13. 13 Motion Energy Models An alternative way to think about 2D motion detection involves using spatio-temporal frequency filters. This type of model relies on filters that are combined to detect a certain range of spatial and temporal frequencies. Various people have developed versions of these models: van Santen & Sperling Watson & Ahumada Adelson & Bergen

14. 14 Motion as orientation In x-t space, motion is an oriented line. The slant depends on speed. In x-y-t space, motion becomes an oriented slab within a volume. x-t x-y-t

15. 15 Orientation Detectors in Space- Time Filters oriented in space time can detect a moving stimulus. The orientation of the filter relates to its preferred speed of motion. These filters can detect sampled motion as well. Oriented Spatio-temporal filters:

16. 16 Separable Spatio-temporal filters A Spatio-temporal filter can be created as the product between a spatial filter and a temporal filter. Spatial impulse response = H S (x) Temporal impulse response = H T (t) Spatio-temporal impulse response: H ST (x, t) = H S (x)H T (t)

17. 17 Response to a Moving Edge t1t1 t2t2 t3t3 There is little response at t 1 and t 3. There is largest response at t 2 during the edge motion

18. 18 Oriented spatio-temporal filters The previous filter was not selective for direction of motion. We can develop an oriented filter that is selective for direction, by creating a spatio-temporal Gabor filter: - + - Filter selective for leftward motion

19. 19 Response of oriented filter Non-oriented Oriented Moving edge stimulus FilterResponse

20. 20 Problems with Gabor filter The Gabor filter by itself results in several problems: 1)It is phase sensitive: It depends on a particular alignment of the pattern with the filter at a given time. (The response to a drifting sinewave is an oscillation). 2)The sign of the response depends on the stimulus contrast (e.g. white on black gives opposite response to black on white).

21. 21 Solution: Motion Energy Motion energy filters are constructed with 2 gabor filters, one of which uses a sine and the other uses a cosine (a "quadrature pair"). If you square the outputs of the gabors and sum, the result is motion energy.

22. 22 Motion Energy Responses With motion energy filters: The response is always positive. The response is the same for a black-white edge as for a white- black edge. The response to motion is independent of contrast. The response is constant for a drifting sinewave.