1. Course 11 Optical Flow

2. 1. Concept ----- Observe the scene by moving viewer. ----- Optical flow provides a clue to recover the motion. 2. Constraint equation of optical flow image observed from a scene at time t: at t+dt for the same scene: If image intensity change is small in time interval, we have: i.e.

3. By chain rule of differentiation, Denote optical flow and image gradient we have:

4. This equation is called constraint equation of optical flow. From a pair of images of a small time interval, f x, f y and f t can be calculated. However, u, v cannot be solved from only constraint equation of optical flow. This is called aperture problem. 3. Solve for optical flow Considering continuity constraint of optical flow:

5. Using variation method, we get Let i.e. It yields that:

6. Solve the equation, get: Since and need u and v and their neighbor pixels to calculate, we can use iterative method to solve them.

8. 4. Understand optical flow 1) Focus of Expansion (FOE). When one does translational motion along a fixed direction, the world seems to be flowing out of one particular retinal point. This point is called focus of expansion (FOE).

9. Remark: (1) FOE is a point in image plane. (2) FOE appears only for tranlational motion. Considering a 3D point at (x 0, y 0, z 0 ) initially and observer is moving at velocity. After a time interval t, the 3D position of the point will be at (x 0 +w x t, y 0 +w y t, z 0 +w z t) and the image point is:

10. When time, we get FOE: 2) Time–to–adjacency relation [Ballard, Computer Vision] Let p be a 3D point at a translational moving object; be the 3D speed of p in z-direction. Let D be the distance from FOE to the image point along straight flow; be the optic flow speed of the image point.

11. Then: thus: Recalling the formula of perspective projection, we have

12. This indicates that if we know the 3D speed of an object in z- direction, the distance from FOE to the corresponding image point and the flow speed in the image plane, the 3D position of the object can be computed. 3) Using Optical flow to segment image of moving objects ----- Continuing property of optical flow. ----- FOE.