Download presentation

Presentation is loading. Please wait.

Published byClarissa claudia Ott Modified about 1 year ago

1
Optical Flow Methods CISC 489/689 Spring 2009 University of Delaware

2
Outline Review of Optical Flow Constraint, Lucas- Kanade, Horn and Schunck Methods Lucas-Kanade Meets Horn and Schunck 3D Regularization Techniques for solving optical flow Confidence Measures in Optical Flow

3
Optical Flow Constraint

4
Interpretation Constraint Line

5
Lucas-Kanade Method * =

6
Local Method, window based Cannot solve for optical flow everywhere Robust to noise Figures from Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods ANDR´ES BRUHN AND JOACHIM WEICKERT, 2005

7
Dense optical Flow Lacks Smoothness Figures from Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods ANDR´ES BRUHN AND JOACHIM WEICKERT, 2005

8
Horn and Schunck Method Euler-Lagrange Equations

9
Horn and Schunck Method Figures from Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods ANDR´ES BRUHN AND JOACHIM WEICKERT, 2005 Global Method Estimates flow everywhere Sensitive to noise Oversmooths the edges

10
Why combine them? Need dense flow estimate Robust to noise Preserve discontinuities

11
Combining the two…

12
Combined Local Global Method Euler-Lagrange Equations Average Error Standard Deviation Lucas&Kanade 4.3 (density 35%) Horn&Schunk 9.816.2 Combining local and global 4.27.7 Table: Courtesy - Darya Frolova, Recent progress in optical flow

13
Preserving discontinuities Gaussian Window does not preserve discontinuities Solutions – Use bilateral filtering – Add gradient constancy

14
Bilateral support window Images: Courtesy, Darya Frolova, Recent progress in optical flow

15
Robust statistics – simple example Find “best” representative for the set of numbers L2: L1: xixi Influence of x i on E : x i → x i + ∆ Outliers influence the most proportional to Majority decides equal for all x i Slide: Courtesy - Darya Frolova, Recent progress in optical flow

16
Robust statistics many ordinary people a very rich man Oligarchy Votes proportional to the wealth Democracy One vote per person wealth like in L2 norm minimization like in L1 norm minimization Slide: Courtesy - Darya Frolova, Recent progress in optical flow

17
Combination of two flow constraints robust: L1 robust in presence of outliers – non-smooth: hard to analyze usual: L2 easy to analyze and minimize – sensitive to outliers robust regularized smooth: easy to analyze robust in presence of outliers ε [A. Bruhn, J. Weickert, 2005] Towards ultimate motion estimation: Combining highest accuracy with real-time performance Slide: Courtesy - Darya Frolova, Recent progress in optical flow

18
Robust statistics

19
3D Regularization accounted for spatial regularization If velocities do not change suddenly with time, can we regularize in time as well?

20
3D Regularization

21
Multiresolution estimation 21 image I image J Gaussian pyramid of image 1Gaussian pyramid of image 2 Image 2 image 1 run iterative estimation warp & upsample......

22
Multi-resolution Lucas Kanade Algorithm Compute Iterative LK at highest level For Each Level i Take flow u(i-1), v(i-1) from level i-1 Upsample the flow to create u*(i), v*(i) matrices of twice resolution for level i. Multiply u*(i), v*(i) by 2 Compute I t from a block displaced by u*(i), v*(i) Apply LK to get u’(i), v’(i) (the correction in flow) Add corrections u’(i), v’(i) to obtain the flow u(i), v(i) at the i th level, i.e., u(i)=u*(i)+u’(i), v(i)=v*(i)+v’(i)

23
Comparison of errors For Yosemite sequence with clouds Table: Courtesy - Darya Frolova, Recent progress in optical flow

24
Solving the system How to solve? 2 components of success: fast convergence good initial guess Start with some initial guess and apply some iterative method

25
smoothing Relaxation schemes have smoothing property: Only neighboring pixels are coupled in relaxation scheme It may take thousands of iterations to propagate information to large distance...... Relaxation smoothes the error

26
Relaxation smoothes the error Examples 2D case: 1D case: Error of initial guess Error after 5 relaxation Error after 15 relaxations

27
Idea: coarser grid On a coarser grid low frequencies become higher Hence, relaxations can be more effective initial grid – fine grid coarse grid – we take every second point

28
Multigrid 2-Level V-Cycle 1. Iterate ⇒ error becomes smooth 2. Transfer error equation to the coarse level ⇒ low frequencies become high 3. Solve for the error on the coarse level ⇒ good error estimation 4. Transfer error to the fine level 5. Correct the previous solution 6. Iterate ⇒ remove interpolation artifacts

29
make iteration process faster (on the coarse grid we can effectively minimize the error) obtain better initial guess (solve directly on the coarsest grid) Coarse grid - advantages Coarsening allows: go to the coarsest grid solve here the equation to find interpolate to the finer grid

30
Multigrid approach – Full scheme

31
Confidence Metric Intrinsic in Local Methods How to evaluate for global methods? – Edge strength? Doesn’t work (Barron et al.,1994)

32
Confidence Metric Histogram of error contribution Error Number of pixels

33
Confidence Metric

34
More Results

36
Further Reading Combining the advantages of local and global optic flow methods (“Lucas/Kanade meets Horn/Schunck”) A. Bruhn, J. Weickert, C. Schnörr, 2002 - 2005 High accuracy optical flow estimation based on a theory for warping T. Brox, A. Bruhn, N. Papenberg, J. Weickert, 2004 - 2005 Real-Time Optic Flow Computation with Variational Methods A. Bruhn, J. Weickert, C. Feddern, T. Kohlberger, C. Schnörr, 2003 - 2005 Towards ultimate motion estimation: Combining highest accuracy with real-time performance. A. Bruhn, J. Weickert, 2005 Bilateral filtering-based optical flow estimation with occlusion detection. J.Xiao, H.Cheng, H.Sawhney, C.Rao, M.Isnardi, 2006

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google