Presentation is loading. Please wait.

Presentation is loading. Please wait.

Optical Flow Methods CISC 489/689 Spring 2009 University of Delaware.

Similar presentations


Presentation on theme: "Optical Flow Methods CISC 489/689 Spring 2009 University of Delaware."— Presentation transcript:

1 Optical Flow Methods CISC 489/689 Spring 2009 University of Delaware

2 Outline Review of Optical Flow Constraint, Lucas- Kanade, Horn and Schunck Methods Lucas-Kanade Meets Horn and Schunck 3D Regularization Techniques for solving optical flow Confidence Measures in Optical Flow

3 Optical Flow Constraint

4 Interpretation Constraint Line

5 Lucas-Kanade Method * =

6 Local Method, window based Cannot solve for optical flow everywhere Robust to noise Figures from Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods ANDR´ES BRUHN AND JOACHIM WEICKERT, 2005

7 Dense optical Flow Lacks Smoothness Figures from Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods ANDR´ES BRUHN AND JOACHIM WEICKERT, 2005

8 Horn and Schunck Method Euler-Lagrange Equations

9 Horn and Schunck Method Figures from Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods ANDR´ES BRUHN AND JOACHIM WEICKERT, 2005 Global Method Estimates flow everywhere Sensitive to noise Oversmooths the edges

10 Why combine them? Need dense flow estimate Robust to noise Preserve discontinuities

11 Combining the two…

12 Combined Local Global Method Euler-Lagrange Equations Average Error Standard Deviation Lucas&Kanade 4.3 (density 35%) Horn&Schunk Combining local and global Table: Courtesy - Darya Frolova, Recent progress in optical flow

13 Preserving discontinuities Gaussian Window does not preserve discontinuities Solutions – Use bilateral filtering – Add gradient constancy

14 Bilateral support window Images: Courtesy, Darya Frolova, Recent progress in optical flow

15 Robust statistics – simple example Find “best” representative for the set of numbers L2: L1: xixi Influence of x i on E : x i → x i + ∆ Outliers influence the most proportional to Majority decides equal for all x i Slide: Courtesy - Darya Frolova, Recent progress in optical flow

16 Robust statistics many ordinary people a very rich man Oligarchy Votes proportional to the wealth Democracy One vote per person wealth like in L2 norm minimization like in L1 norm minimization Slide: Courtesy - Darya Frolova, Recent progress in optical flow

17 Combination of two flow constraints robust: L1 robust in presence of outliers – non-smooth: hard to analyze usual: L2 easy to analyze and minimize – sensitive to outliers robust regularized smooth: easy to analyze robust in presence of outliers ε [A. Bruhn, J. Weickert, 2005] Towards ultimate motion estimation: Combining highest accuracy with real-time performance Slide: Courtesy - Darya Frolova, Recent progress in optical flow

18 Robust statistics

19 3D Regularization accounted for spatial regularization If velocities do not change suddenly with time, can we regularize in time as well?

20 3D Regularization

21 Multiresolution estimation 21 image I image J Gaussian pyramid of image 1Gaussian pyramid of image 2 Image 2 image 1 run iterative estimation warp & upsample......

22 Multi-resolution Lucas Kanade Algorithm Compute Iterative LK at highest level For Each Level i Take flow u(i-1), v(i-1) from level i-1 Upsample the flow to create u*(i), v*(i) matrices of twice resolution for level i. Multiply u*(i), v*(i) by 2 Compute I t from a block displaced by u*(i), v*(i) Apply LK to get u’(i), v’(i) (the correction in flow) Add corrections u’(i), v’(i) to obtain the flow u(i), v(i) at the i th level, i.e., u(i)=u*(i)+u’(i), v(i)=v*(i)+v’(i)

23 Comparison of errors For Yosemite sequence with clouds Table: Courtesy - Darya Frolova, Recent progress in optical flow

24 Solving the system How to solve? 2 components of success: fast convergence good initial guess Start with some initial guess and apply some iterative method

25 smoothing Relaxation schemes have smoothing property: Only neighboring pixels are coupled in relaxation scheme It may take thousands of iterations to propagate information to large distance Relaxation smoothes the error

26 Relaxation smoothes the error Examples 2D case: 1D case: Error of initial guess Error after 5 relaxation Error after 15 relaxations

27 Idea: coarser grid On a coarser grid low frequencies become higher Hence, relaxations can be more effective initial grid – fine grid coarse grid – we take every second point

28 Multigrid 2-Level V-Cycle 1. Iterate ⇒ error becomes smooth 2. Transfer error equation to the coarse level ⇒ low frequencies become high 3. Solve for the error on the coarse level ⇒ good error estimation 4. Transfer error to the fine level 5. Correct the previous solution 6. Iterate ⇒ remove interpolation artifacts

29 make iteration process faster (on the coarse grid we can effectively minimize the error) obtain better initial guess (solve directly on the coarsest grid) Coarse grid - advantages Coarsening allows: go to the coarsest grid solve here the equation to find interpolate to the finer grid

30 Multigrid approach – Full scheme

31 Confidence Metric Intrinsic in Local Methods How to evaluate for global methods? – Edge strength? Doesn’t work (Barron et al.,1994)

32 Confidence Metric Histogram of error contribution Error Number of pixels

33 Confidence Metric

34 More Results

35

36 Further Reading Combining the advantages of local and global optic flow methods (“Lucas/Kanade meets Horn/Schunck”) A. Bruhn, J. Weickert, C. Schnörr, High accuracy optical flow estimation based on a theory for warping T. Brox, A. Bruhn, N. Papenberg, J. Weickert, Real-Time Optic Flow Computation with Variational Methods A. Bruhn, J. Weickert, C. Feddern, T. Kohlberger, C. Schnörr, Towards ultimate motion estimation: Combining highest accuracy with real-time performance. A. Bruhn, J. Weickert, 2005 Bilateral filtering-based optical flow estimation with occlusion detection. J.Xiao, H.Cheng, H.Sawhney, C.Rao, M.Isnardi, 2006


Download ppt "Optical Flow Methods CISC 489/689 Spring 2009 University of Delaware."

Similar presentations


Ads by Google