1. ICG Professor Horst Cerjak, 19.12.2005 1 Thomas Pock Variational Methods for 3D Reconstruction Thomas Pock 1, Chrisopher Zach 2 and Horst Bischof 1 1 Institute for Computer Graphics and Vision, TU Graz 2 VRVis Forschungs GmbH E-mail: {pock, zach, bischof}@icg.tugraz.at

2. ICG Professor Horst Cerjak, 19.12.2005 2 Thomas Pock Variational Methods for 3D Reconstruction Outline Introduction to Variational Methods (I) Variational Stereo (II) Variational Optical Flow (III) Variational Fusion of Depth Images

3. ICG Professor Horst Cerjak, 19.12.2005 3 Thomas Pock Variational Methods for 3D Reconstruction Introduction to Variational Methods Computer vision deals with inverse problems Bayesian framework is often used to estimate the unknown quantities Equivalent to minimizing appropriate energy functionals (neg. log likelihood) This directly leads to Variational Methods

4. ICG Professor Horst Cerjak, 19.12.2005 4 Thomas Pock Variational Methods for 3D Reconstruction (I) Variational Stereo Typical stereo problems: –Occlusions –Large untextured areas –Varying radiance

5. ICG Professor Horst Cerjak, 19.12.2005 5 Thomas Pock Variational Methods for 3D Reconstruction Simple Correlation Methods Problems –Huge amount of outliers in untextured areas –Weak performance at depth discontinuities Observations –Depth discontinuities often coincides with image edges –Robust estimation in untextured areas

6. ICG Professor Horst Cerjak, 19.12.2005 6 Thomas Pock Variational Methods for 3D Reconstruction Our Approach Joint color edge detection and robust depth estimation using a Mumford-Shah Energy Functional d z (z,d) = argmin { E }

7. ICG Professor Horst Cerjak, 19.12.2005 7 Thomas Pock Variational Methods for 3D Reconstruction E initial 28.8 %28.1 %25.1 %14.3 % E final 18.3 %3.45 %11.2 %7.52 % Rank12446 Results

8. ICG Professor Horst Cerjak, 19.12.2005 8 Thomas Pock Variational Methods for 3D Reconstruction Real Dataset Synthesized view

9. ICG Professor Horst Cerjak, 19.12.2005 9 Thomas Pock Variational Methods for 3D Reconstruction (II) Variational Optical Flow Optical Flow plays a major task of biological and artificial visual systems Relates the motion of pixels between different image frames Highly ill-posed inverse problem. More unknowns than equations (Optical Flow Constraint)

10. ICG Professor Horst Cerjak, 19.12.2005 10 Thomas Pock Variational Methods for 3D Reconstruction Our Approach We employ a robust energy functional consisting of –Total Variation Regularization ( Discontinuity preserving) –Robust L 1 data fidelity norm (Robust against noise, occlusions, …) We have developed an novel minimization algorithm Can be efficiently implemented on graphics hardware Image resolution 50 It100200 160x12059.829.915.0 320x24019.69.95.0 640x4805.52.81.4 Performance (fps):

11. ICG Professor Horst Cerjak, 19.12.2005 11 Thomas Pock Variational Methods for 3D Reconstruction Results

12. ICG Professor Horst Cerjak, 19.12.2005 12 Thomas Pock Variational Methods for 3D Reconstruction Application to Stereo

13. ICG Professor Horst Cerjak, 19.12.2005 13 Thomas Pock Variational Methods for 3D Reconstruction (III) Variational Fusion of Depth Images Generation of 3D Models from multiple views Huge amount of Outliers but a lot of redundancy Two Requirements: –Robust against outliers –Natural Regularization of the 3D Model

14. ICG Professor Horst Cerjak, 19.12.2005 14 Thomas Pock Variational Methods for 3D Reconstruction Our Approach Conversion of the depth images into 3D truncated signed distance fields (f i ) Assignment of varying confidence (w i ) Minimization of an energy functional consisting of –Total Variation Regularization –L 1 data fidelity term

15. ICG Professor Horst Cerjak, 19.12.2005 15 Thomas Pock Variational Methods for 3D Reconstruction Results

16. ICG Professor Horst Cerjak, 19.12.2005 16 Thomas Pock Variational Methods for 3D Reconstruction Thank You !