Download presentation

Presentation is loading. Please wait.

Published byClarissa claudia Ott Modified over 4 years ago

1
**CISC 489/689 Spring 2009 University of Delaware**

Optical Flow Methods CISC 489/689 Spring 2009 University of Delaware

2
Outline Review of Optical Flow Constraint, Lucas-Kanade, Horn and Schunck Methods Lucas-Kanade Meets Horn and Schunck 3D Regularization Techniques for solving optical flow Confidence Measures in Optical Flow

3
**Optical Flow Constraint**

4
Interpretation Constraint Line

5
Lucas-Kanade Method * =

6
**Lucas-Kanade Method Local Method, window based**

Cannot solve for optical flow everywhere Robust to noise Figures from Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods ANDR´ES BRUHN AND JOACHIM WEICKERT, 2005

7
**Dense optical Flow Lacks Smoothness**

Figures from Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods ANDR´ES BRUHN AND JOACHIM WEICKERT, 2005

8
**Horn and Schunck Method**

Euler-Lagrange Equations

9
**Horn and Schunck Method**

Global Method Estimates flow everywhere Sensitive to noise Oversmooths the edges Figures from Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods ANDR´ES BRUHN AND JOACHIM WEICKERT, 2005

10
**Why combine them? Need dense flow estimate Robust to noise**

Preserve discontinuities

11
Combining the two…

12
**Combined Local Global Method**

Euler-Lagrange Equations Average Error Standard Deviation Lucas&Kanade 4.3 (density 35%) Horn&Schunk 9.8 16.2 Combining local and global 4.2 7.7 Table: Courtesy - Darya Frolova, Recent progress in optical flow

13
**Preserving discontinuities**

Gaussian Window does not preserve discontinuities Solutions Use bilateral filtering Add gradient constancy

14
**Bilateral support window**

Images: Courtesy, Darya Frolova, Recent progress in optical flow

15
**Robust statistics – simple example**

Find “best” representative for the set of numbers xi L2: L1: Influence of xi on E: xi → xi + ∆ proportional to equal for all xi Outliers influence the most Majority decides Slide: Courtesy - Darya Frolova, Recent progress in optical flow

16
**Robust statistics many ordinary people a very rich man wealth**

Oligarchy Democracy ordinary people – 1 vote a rich man – many votes If you know what to do – choose oligarchy, if don’t know - democracy Votes proportional to the wealth One vote per person like in L2 norm minimization like in L1 norm minimization Slide: Courtesy - Darya Frolova, Recent progress in optical flow

17
**Combination of two flow constraints**

usual: L2 easy to analyze and minimize sensitive to outliers robust: L1 robust in presence of outliers non-smooth: hard to analyze robust regularized smooth: easy to analyze robust in presence of outliers ε [A. Bruhn, J. Weickert, 2005] Towards ultimate motion estimation: Combining highest accuracy with real-time performance Slide: Courtesy - Darya Frolova, Recent progress in optical flow

18
Robust statistics

19
**3D Regularization accounted for spatial regularization**

If velocities do not change suddenly with time, can we regularize in time as well?

20
3D Regularization

21
**Multiresolution estimation**

Gaussian pyramid of image 1 Gaussian pyramid of image 2 Image 2 image 1 run iterative estimation warp & upsample run iterative estimation . image J image I

22
**Multi-resolution Lucas Kanade Algorithm**

Compute Iterative LK at highest level For Each Level i Take flow u(i-1), v(i-1) from level i-1 Upsample the flow to create u*(i), v*(i) matrices of twice resolution for level i. Multiply u*(i), v*(i) by 2 Compute It from a block displaced by u*(i), v*(i) Apply LK to get u’(i), v’(i) (the correction in flow) Add corrections u’(i), v’(i) to obtain the flow u(i), v(i) at the ith level, i.e., u(i)=u*(i)+u’(i), v(i)=v*(i)+v’(i)

23
**Comparison of errors For Yosemite sequence with clouds**

Table: Courtesy - Darya Frolova, Recent progress in optical flow

24
**Solving the system How to solve? Start with some initial guess**

and apply some iterative method fast convergence good initial guess 2 components of success:

25
**Relaxation smoothes the error**

Relaxation schemes have smoothing property: It may take thousands of iterations to propagate information to large distance oscillatory modes of the error are eliminated effectively, but smooth modes are damped slowly Only neighboring pixels are coupled in relaxation scheme

26
**Relaxation smoothes the error Examples**

1D case: 2D case: Error of initial guess Error after 5 relaxation Error after 15 relaxations

27
**Idea: coarser grid initial grid – fine grid**

On a coarser grid low frequencies become higher Hence, relaxations can be more effective coarse grid – we take every second point

28
**Multigrid 2-Level V-Cycle**

5. Correct the previous solution 6. Iterate ⇒ remove interpolation artifacts 1. Iterate ⇒ error becomes smooth 2. Transfer error equation to the coarse level ⇒ low frequencies become high 4. Transfer error to the fine level 3. Solve for the error on the coarse level ⇒ good error estimation

29
**Coarse grid - advantages**

Coarsening allows: make iteration process faster (on the coarse grid we can effectively minimize the error) obtain better initial guess (solve directly on the coarsest grid) go to the coarsest grid interpolate to the finer grid solve here the equation to find

30
**Multigrid approach – Full scheme**

31
**Confidence Metric Intrinsic in Local Methods**

How to evaluate for global methods? Edge strength? Doesn’t work (Barron et al.,1994)

32
**Confidence Metric Histogram of error contribution Number of pixels**

33
Confidence Metric

34
More Results

35
More Results

36
Further Reading Combining the advantages of local and global optic flow methods (“Lucas/Kanade meets Horn/Schunck”) A. Bruhn, J. Weickert, C. Schnörr, High accuracy optical flow estimation based on a theory for warping T. Brox, A. Bruhn, N. Papenberg, J. Weickert, Real-Time Optic Flow Computation with Variational Methods A. Bruhn, J. Weickert, C. Feddern, T. Kohlberger, C. Schnörr, Towards ultimate motion estimation: Combining highest accuracy with real-time performance. A. Bruhn, J. Weickert, 2005 Bilateral filtering-based optical flow estimation with occlusion detection. J.Xiao, H.Cheng, H.Sawhney, C.Rao, M.Isnardi, 2006

Similar presentations

OK

Optical Flow Digital Photography CSE558, Spring 2003 Richard Szeliski (notes cribbed from P. Anandan)

Optical Flow Digital Photography CSE558, Spring 2003 Richard Szeliski (notes cribbed from P. Anandan)

© 2019 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google