Presentation on theme: "Fast and Accurate Optical Flow Estimation Primal-Dual Schemes and Second Order Priors Thomas Pock and Daniel Cremers CVPR Group, University of Bonn Collaborators:"— Presentation transcript:
Fast and Accurate Optical Flow Estimation Primal-Dual Schemes and Second Order Priors Thomas Pock and Daniel Cremers CVPR Group, University of Bonn Collaborators: Christopher Zach, Markus Unger, Werner Trobin, and Horst Bischof
Variational Optical Flow – Short History Horn and Schunck 1981 Black and Anadan, Cohen 1993 Aubert 20002004 Brox et al. 2006 Bruhn et al.
Outline Model of Horn and Schunck TV-L 1 Model Fast Numerical Scheme Parallel Implementation 2 nd order Prior
The Model of Horn and Schunck  Regularization TermData Term (OFC) + Convex + Easy to solve - Does not allow for sharp edges in the solution - Sensitive to outliers violating the OFC  Horn and Schunck. Determinig Optical Flow. Artificial Intelligence, 1981
Can we do better? Replace quadratic functions by L 1 – norms Done by Cohen, Aubert, Brox, Bruhn,... +Allows for discontinuities in the flow field +Robust to some extent to outliers in the OFC +Still convex - Much harder to solve
How can we minimize this functional ? Compute Euler-Lagrange Equations Non-linear, non-smooth,...
Standard Approach Replace L 1 – norm by regularized variants (Charbonnier function) Example: Small epsilon: Nearly degenerated Large espilon: Smears edges
Our Approach(1) Introduce auxiliary variables and constraints Quadratic penalty
Our Approach(2) What do we gain? We solve a sequence of simpler problems Algorithm: 1.For fixed (u´,v´), solve for(u,v) using Chambolles algorithm 2.For fixed (u,v), solve for (u´,v´) using a 1D shrinkage formula 3.Goto 1. until convergence  Rudin, Osher and Fatemi. Nonlinear Total Variation Based Noise Removal Algorithms, 1992  Zach, Pock and Bischof. A Duality Based Algorithm for Realtime TV-L 1 Optical Flow, DAGM 2007  Chambolle. An Algorithm for Total Variation Minimization, 2004. 1D Problem ROF Model 
Implementation Numerical scheme can be easily parallelized We use state-of-the-art GPUs
Performance Evaluation Image SizeFrames per Second 128x128192 256x256108 512x51236 TV-L 1 Optical Flow Implemented in CUDA 2.0 Computed on Nvidia GeForce GTX 280 25 Overall Iterations (5 Chambolle Iterations)
2 nd order Prior TV regularization favors piecewise constant flow fields (frontoparallel motion) Extension to piecewise affine flow fields? Approach of Cremers et al.  – Fixed number of regions Approach of Nir et al.  – Over-parametrized optical flow Our approach  – 2 nd order derivatives to regularize flow field  Cremers and Soatto, Motion Competition: A Variational Framework for Piecwise Parametric Motion Segmentation.  Nir, Bruckstein and Kimmel, Over-Parameterized Variational Optical Flow, IJCV 2007  Trobin, Pock, Cremers and Bischof, An Unbiased Second-Order prPior for High-Accuracy Motion Estimation, DAGM 2008
2 nd -L 1 Optical Flow 2 nd order derivatives are not orthogonal We use a transformation due to Danielsson  Optimization – Similar strategy to TV-L 1 – 4 th order PDE  Danielsson and Lin, Efficient Detection of Second-Degree Variations in 2D and 3D Images, 2001.
Why does it allow for discontinuities ? 1.0 0.11 1.0 0.01 Total Variation has no bias against discontinuities
Evaluation of Optical Flow Methods  Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M., Szeliski, R.: A database and evaluation methodology for optical flow. ICCV 2007 Input Images Ground Truth