1. Motion / Optical Flow II Estimation of Motion Field Avneesh Sud

2. Computer Vision - Fall 20012 Outline Motion Field & Optical Flow Constraints Methods of Estimating Motion Field –Differential Techniques Least Squares Horn-Schunck Algorithm Comments Results by Miguel

3. Computer Vision - Fall 20013 Motion Field 2-D projection of velocities of the image points, induced by the relative motion between camera and scene –Not directly measurable from an image

4. Computer Vision - Fall 20014 Optical Flow A vector field subject to Image Brightness Constancy Equation (IBCE) Apparent motion of the image brightness pattern

5. Computer Vision - Fall 20015 Optical Flow Vs. Motion Field Optical flow does not always correspond to motion field Optical flow is an approximation of the motion field. The error is small at points with high spatial gradient under some simplifying assumptions (Trucco p195)

6. Computer Vision - Fall 20016 Correspondence between points on isobrightness contours? A constant patch of uniform brightness – multiple optical flow solutions Ambiguity in Local Optical Flow – Use additional constraints !

7. Computer Vision - Fall 20017 IBCE Revisited Also known as the Horn and Schunck optical flow constraint equation Assume the image intensity of each visible scene point is unchanging over time

8. Computer Vision - Fall 20018 Constraint corresponds to a line in velocity space Given local info, can determine component of optical flow vector only in direction of brightness gradient Aperture Problem u v (E x,E y ) Constraint line

9. Computer Vision - Fall 20019 Estimating Motion Field Differential techniques : based on spatial & temporal variations of the image at all pixels Matching (feature-based) techniques : rely on special image points (features) and track them through frames

10. Computer Vision - Fall 200110 Differential Techniques : Least Squares Optical Flow Algorithm (Trucco, p196) –For each pixel p Must satisfy (  E)v + E t = 0 Assumption : This equation holds in the neighborhood of p with constant v Write this equation for a small (typically 5x5) patch centered at p Then we find least square fit of v - this is the calculated optical flow for pixel p

11. Computer Vision - Fall 200111 Least Squares : Assumptions Assumed that ICBE holds in the neighborhood of p with constant v In case of rigid motion, the motion field of a moving plane is a quadratic polynomial in the coordinates (x, y, f) of the image points. ( Trucco p 187) –Therefore, if the object is smooth & rigid, we can assume the motion field varies smoothly

12. Computer Vision - Fall 200112 Differential Techniques : Horn- Schunck Algorithm Optical flow constraint equation gives the component in direction of brightness gradient : Additional Constraint : smoothness of optical flow! Neighboring surface points of a rigid object have approximately same local displacement vectors

13. Computer Vision - Fall 200113 Horn-Schunck Algorithm Two criteria: –Optical flow is smooth, F s (u,v) –Small error in optical flow constraint equation, F h (u,v) Minimize a combined error functional F c (u,v) = F s (u,v) + λ F h (u,v) λ is a weighting parameter Variation calculus gives a pair of second order differential equations that can be solved iteratively

14. Computer Vision - Fall 200114 Horn-Schunck Algorithm : Discrete Case Derivatives (and error functionals) are approximated by difference operators Leads to an iterative solution:

15. Computer Vision - Fall 200115 Intuition of the Iterative Scheme u v (E x,E y ) Constraint line (u,v)(u,v) The new value of (u,v) at a point is equal to the average of surrounding values minus an adjustment in the direction of the brightness gradient

16. Computer Vision - Fall 200116 Horn - Schunck Algorithm

17. Computer Vision - Fall 200117 Horn-Schunck Algorithm begin for j := 1 to N do for I:= 1 to M do begin calculate the values E x (i,j,t), E y (i,j,t) and E t (i,j,t) using a selected approx formula initialize the values u(I,j) and v(i,j) to zero end {for} choose a suitable weighting value choose a suitable number n 0  1 of iterations n := 1 while n  n 0 do begin for j := 1 to N do for i := 1 to M do begin compute u, v,  update u(i,j), v(i,j) end {for} n := n + 1 end {while} end

18. Computer Vision - Fall 200118 Comments There are reliable methods for estimating optical flow. Optical flow is a vector field, from which the motion field can be estimated under certain conditions. Horn – Schunk Algorithm as presented does not handle discontinuities (silhouettes) well. Some real-world results!

19. Computer Vision - Fall 200119 References Introductory Techniques for 3-D Computer Vision, Emanuele Trucco and Allessandro Verri, Prentice Hall, 1998. Chapter 8 Robot Vision, B.K.P. Horn, MIT Press 1986. Chapter 12 Computer Vision: Three-Dimensional Data from Images, Reinhard Klette, Karsten Schluns, Andreas Koschan, Springer 1998. Topic 5.2

20. Computer Vision - Fall 200120 References MikeTalk: A Talking Facial Display Based on Morphing Visemes, Tony Ezzat and Tomaso Poggio, Proceedings of the Computer Animation Conference Philadelphia, PA, June 1998 Optical Flow - CVonline: Motion and Time Sequence Analysis (http://www.dai.ed.ac.uk/CVonline/motion.htm) (http://www.dai.ed.ac.uk/CVonline/motion.htm)