1. Fast and Accurate Optical Flow Estimation
Primal-Dual Schemes and
Second Order Priors
Thomas Pock and Daniel Cremers
CVPR Group, University of Bonn
Collaborators: Christopher Zach, Markus Unger, Werner Trobin, and Horst Bischof

2. Variational Optical Flow – Short History
1981
1993
2000
2004
2006
Horn and Schunck
Black and Anadan, Cohen
Aubert
Brox et al.
Bruhn et al.

4. Outline Model of Horn and Schunck TV-L1 Model Fast Numerical Scheme
Parallel Implementation
2nd order Prior

5. The Model of Horn and Schunck [1]
Regularization Term
Data Term (OFC)
+ Convex
+ Easy to solve
- Does not allow for sharp edges in the solution
- Sensitive to outliers violating the OFC
[1] Horn and Schunck. Determinig Optical Flow. Artificial Intelligence, 1981

6. Can we do better? Replace quadratic functions by L1 – norms
Done by Cohen, Aubert, Brox, Bruhn, ...
+Allows for discontinuities in the flow field
+Robust to some extent to outliers in the OFC
+Still convex
- Much harder to solve

7. How can we minimize this functional ?
Compute Euler-Lagrange Equations
Non-linear, non-smooth, ...

8. Standard Approach
Replace L1 – norm by regularized variants (Charbonnier function)
Example:
Small epsilon: Nearly degenerated
Large espilon: Smears edges

9. Our Approach(1) Introduce auxiliary variables and constraints
Quadratic penalty

10. Our Approach(2) What do we gain?
We solve a sequence of simpler problems
1D Problem
ROF Model [2]
Algorithm[3]:
For fixed (u´,v´), solve for(u,v) using Chambolle‘s algorithm[4]
For fixed (u,v), solve for (u´,v´) using a 1D shrinkage formula
Goto 1. until convergence
[2] Rudin, Osher and Fatemi. Nonlinear Total Variation Based Noise Removal Algorithms, 1992
[3] Zach, Pock and Bischof. A Duality Based Algorithm for Realtime TV-L1 Optical Flow, DAGM 2007
[4] Chambolle. An Algorithm for Total Variation Minimization, 2004.

11. Implementation Numerical scheme can be easily parallelized
We use state-of-the-art GPUs

12. Performance Evaluation
TV-L1 Optical Flow Implemented in CUDA 2.0
Computed on Nvidia GeForce GTX 280
25 Overall Iterations (5 Chambolle Iterations)
Image Size
Frames per Second
128x128
192
256x256
108
512x512 36

13. Results for TV-L1
Input Image:
Ground Truth:
Our Results:

14. 2nd order Prior
TV regularization favors piecewise constant flow fields (frontoparallel motion)
Extension to piecewise affine flow fields?
Approach of Cremers et al. [5]
Fixed number of regions
Approach of Nir et al. [6]
Over-parametrized optical flow
Our approach [7]
2nd order derivatives to regularize flow field
[5] Cremers and Soatto, Motion Competition: A Variational Framework for Piecwise Parametric Motion Segmentation.
[6] Nir, Bruckstein and Kimmel, Over-Parameterized Variational Optical Flow, IJCV 2007
[7] Trobin, Pock, Cremers and Bischof, An Unbiased Second-Order prPior for High-Accuracy
Motion Estimation, DAGM 2008

15. 2nd-L1 Optical Flow 2nd order derivatives are not orthogonal
We use a transformation due to Danielsson [8]
Optimization
Similar strategy to TV-L1
4th order PDE
[8] Danielsson and Lin, Efficient Detection of Second-Degree Variations in 2D and 3D Images, 2001.

16. Comparison
Ground truth
TV-L1
2nd -L1

17. Results for 2nd-L1
Ground Truth:
Our Results:

18. Conclusion TV-L1 Optical Flow Parallel Implementation 2nd order prior
Fast Numerical Scheme
Parallel Implementation
Realtime Performance
2nd order prior
Piecewise affine motion

19. Recent Application: Tracking

20. Why does it allow for discontinuities ?
1.0
1.0
1.0
0.01
0.11
1.0
Total Variation has no bias against discontinuities

21. Evaluation of Optical Flow Methods
Input Images
Ground Truth
[1] Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M., Szeliski, R.:
A database and evaluation methodology for optical flow. ICCV 2007