1. Graph Cut Algorithms for Binocular Stereo with Occlusions
Hallo erst mal …
Vladimir Kolmogorov, Ramin Zabih

2. Overview: Traditional Stereo Methods
Energy Minimization via Graph Cuts
Stereo with Occlusions
Voxel Labeling Algorithm
Pixel Labeling Algorithm
Results and Conclusions
… outlining the talk
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007

3. Traditional Stereo Methods
Traditional Stereo Problem
pixel correspondences  labeling (disparity)
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007

4. Traditional Stereo Methods Disparity
depth
 disparity ~ depth
ground truth disparity
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007

5. Traditional Stereo Methods Binocular Stereo
goal is to compute pixels correspondences
traditional stereo problem  pixel labeling problem
advantage: can be solved by graph cuts
problem is formulated as energy term
new goal: find the minimizing labeling
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007

6. Traditional Stereo Methods Energy Function
find labeling that minimizes
cost for assigning labels
smoothness term
we assign the label to pixel p when p of image I corresponds to p in I‘
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007

7. Traditional Stereo Methods Energy Function
data cost – gives penalty for different intensities
smoothness term – gives penalty for discontinuities (Potts model)
other models:
absolute distance
quadratic
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007

8. Energy Minimization via Graph Cuts
use for minimizing semimetric terms
Max-flow / Min-Cut
(Ford and Fulkerson Algorithm, Push-Relabel Method)
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007

9. Energy Minimization via Graph Cuts
convex V vs. metric / semimetric
α-β-swap move
α-expansion move: assigning label α to an arbitrary set of pixels
Initial Labeling
α-expansion
α-β-swap
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007

10. Stereo with Occlusions
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007

11. Stereo with Occlusions
treat input symmetrically
scene elements only visible in single view
physically correct scenes  geometric constraints  occlusions  physically possible labelings
introduce constraints in the problem formulation
graph cuts perform unconstrained energy minimization
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007

12. Voxel Labeling Algorithm
discrete scene of voxels
voxel v is active when visible from both cameras
uniqueness constraint – 1:1 correspondence of pixels
„format“ physical possible space with voxels
[722] Voxel Coloring
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007

13. Voxel Labeling Algorithm Energy Function
smoothness term
(Potts model)
matching penalty
(only active voxels)
occlusion penalty
set of occluded pixels
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007

14. Pixel Labeling Algorithm Energy Function
like traditional stereo but for both images e.g. Potts model
active ?
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007

15. Minimizing the Energy
(0=valid, else ∞)
uniqueness
convert constrained into unconstrained minimization problem
write as sum over pairs
form of energy function = standard stereo problem
minimization with α-expansion algorithm
modified definition of α-expansion move for voxel labeling
[490]
binary energy function has to be regular, attained by non-positive data term???
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007

16. Results and Conclusions
Tsukuba ref. image
ground truth
voxel labeling
pixel labeling
traditional s.p.
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007

17. Results and Conclusions
efficient energy minimization polynominal time instead of exponential time
traditional stereo algorithm is faster
pixel labeling better than voxel labeling:
prohibits ‚holes‘ in the scene
allows to use other effective smoothness terms
algorithms can be extended for multiple cameras
Multiple cameras -> voxel coloring [722]
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007

18. Multi-view Stereo via Volumetric Graph Cuts
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007

19. Recent Work
Graph-cut-based stereo matching using image segmentation with symmetrical treatment of occlusions, 2006 TUW
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007

20. Questions? Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007

21. References
M. Bleyer, M. Gelautz, „Graph-cut-based stereo matching using image segmentation with symmetrical treatment of occlusions“, 2007
Y. Boykov, O. Veksler, R. Zabih, „Fast Approximate Energy Minimization via Graph Cuts“, 2001
V. Kolmogorov, R. Zabih, „Graph Cut Algorithms for Binocular Stereo with Occlusions“, 2005
V. Kolmogorov, R. Zabih, „What energy functions can be minimized via graph cuts“, 2004
V. Kolmogorov, R. Zabih, „Generalized multi-camera scene reconstruction using graph cuts“, July 2003
V. Kolmogorov, R. Zabih, „Multi-camera Scene Reconstruction via Graph Cuts“, 2002
S. Seits, C. Dyer, „Photorealistic Scene Reconstruction by Voxel Coloring“, 1997
R.Szeliski, R. Zabih, „An Experimental Comparison of Stereo Algorithms“, 1999
Evelyn Gutschier, Markus Sareika
Graph Cut Algorithms for Binocular Stereo with Occlusions
Math Basics for Vision and Graphis 2007