# Fast and Accurate Optical Flow Estimation

## Presentation on theme: "Fast and Accurate Optical Flow Estimation"— Presentation transcript:

Fast and Accurate Optical Flow Estimation
Primal-Dual Schemes and Second Order Priors Thomas Pock and Daniel Cremers CVPR Group, University of Bonn Collaborators: Christopher Zach, Markus Unger, Werner Trobin, and Horst Bischof

Variational Optical Flow – Short History
1981 1993 2000 2004 2006 Horn and Schunck Black and Anadan, Cohen Aubert Brox et al. Bruhn et al.

Outline Model of Horn and Schunck TV-L1 Model Fast Numerical Scheme
Parallel Implementation 2nd order Prior

The Model of Horn and Schunck [1]
Regularization Term Data Term (OFC) + Convex + Easy to solve - Does not allow for sharp edges in the solution - Sensitive to outliers violating the OFC [1] Horn and Schunck. Determinig Optical Flow. Artificial Intelligence, 1981

Can we do better? Replace quadratic functions by L1 – norms
Done by Cohen, Aubert, Brox, Bruhn, ... +Allows for discontinuities in the flow field +Robust to some extent to outliers in the OFC +Still convex - Much harder to solve

How can we minimize this functional ?
Compute Euler-Lagrange Equations Non-linear, non-smooth, ...

Standard Approach Replace L1 – norm by regularized variants (Charbonnier function) Example: Small epsilon: Nearly degenerated Large espilon: Smears edges

Our Approach(1) Introduce auxiliary variables and constraints

Our Approach(2) What do we gain?
We solve a sequence of simpler problems 1D Problem ROF Model [2] Algorithm[3]: For fixed (u´,v´), solve for(u,v) using Chambolle‘s algorithm[4] For fixed (u,v), solve for (u´,v´) using a 1D shrinkage formula Goto 1. until convergence [2] Rudin, Osher and Fatemi. Nonlinear Total Variation Based Noise Removal Algorithms, 1992 [3] Zach, Pock and Bischof. A Duality Based Algorithm for Realtime TV-L1 Optical Flow, DAGM 2007 [4] Chambolle. An Algorithm for Total Variation Minimization, 2004.

Implementation Numerical scheme can be easily parallelized
We use state-of-the-art GPUs

Performance Evaluation
TV-L1 Optical Flow Implemented in CUDA 2.0 Computed on Nvidia GeForce GTX 280 25 Overall Iterations (5 Chambolle Iterations) Image Size Frames per Second 128x128 192 256x256 108 512x512 36

Results for TV-L1 Input Image: Ground Truth: Our Results:

2nd order Prior TV regularization favors piecewise constant flow fields (frontoparallel motion) Extension to piecewise affine flow fields? Approach of Cremers et al. [5] Fixed number of regions Approach of Nir et al. [6] Over-parametrized optical flow Our approach [7] 2nd order derivatives to regularize flow field [5] Cremers and Soatto, Motion Competition: A Variational Framework for Piecwise Parametric Motion Segmentation. [6] Nir, Bruckstein and Kimmel, Over-Parameterized Variational Optical Flow, IJCV 2007 [7] Trobin, Pock, Cremers and Bischof, An Unbiased Second-Order prPior for High-Accuracy Motion Estimation, DAGM 2008

2nd-L1 Optical Flow 2nd order derivatives are not orthogonal
We use a transformation due to Danielsson [8] Optimization Similar strategy to TV-L1 4th order PDE [8] Danielsson and Lin, Efficient Detection of Second-Degree Variations in 2D and 3D Images, 2001.

Comparison Ground truth TV-L1 2nd -L1

Results for 2nd-L1 Ground Truth: Our Results:

Conclusion TV-L1 Optical Flow Parallel Implementation 2nd order prior
Fast Numerical Scheme Parallel Implementation Realtime Performance 2nd order prior Piecewise affine motion

Recent Application: Tracking

Why does it allow for discontinuities ?
1.0 1.0 1.0 0.01 0.11 1.0 Total Variation has no bias against discontinuities

Evaluation of Optical Flow Methods
Input Images Ground Truth [1] Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M., Szeliski, R.: A database and evaluation methodology for optical flow. ICCV 2007