1. Aleixo Cambeiro Barreiro 광주과학기술원 컴퓨터 비전 연구실
Stereo Matching
Aleixo Cambeiro Barreiro
광주과학기술원 컴퓨터 비전 연구실

2. Index
Abstract
Algorithms used
Results
Conclusion
Bibliography

3. Abstract

4. Stereo Matching
Process of generation of a depth map for one scene having several pictures of it
Achieved by comparing the disparities of the objects’ positions between pixels
Takes into account cameras’ conic fields of vision

5. Goals
In this experiment we intend to compare several dense stereo matching algorithms
To this end, we have considered:
Time elapsed during computation
Results accuracy

6. Algorithms used

7. Pixel-wise Matching
Intensity, Winner Takes All pixel-to-pixel comparison
Base image
Target image
O  Matching pixel  Matching candidates *  Current candidate

8. Patch-wise Matching (I)
Similar to pixel-wise matching
Also takes into account neighbors of the matching pixels to improve accuracy
Base image
Target image
O  Matching pixel  Matching candidates *  Current candidate x  Current neighbor

9. Patch-wise Matching (II)
It can be subdivided according to the way the patches are compared
We have tested the following two comparison methods:
Sum of Absolute Differences (SAD):
Sums differences between corresponding pixels in both patches
This absolute differences sum is used as matching cost
Doesn’t take into account depth differences between pixels in the same patch
Adaptive Support-Weight:
Similar to SAD, but weights are added to intensity differences to take into account depth disparities between pixels in the same patch, according to next equation:
𝑤 𝑝,𝑞 = exp − ∆ 𝑐 𝑝𝑞 𝛾 𝑐 + ∆ 𝑔 𝑝𝑞 𝛾 𝑝

10. Semi-Global Matching (SGM)
Uses pixel-wise matching as a base cost
Adds an one-dimensional, multidirectional smoothness constraint
This generates an energy function that should be minimized
The minimization can be efficiently achieved through dynamic programming
Cost is propagated according to the next equation:
𝐿 𝑟 𝑝,𝑑 =𝐶 𝑝,𝑑 + min 𝐿 𝑟 𝑝−𝑟,𝑑 , 𝐿 𝑟 𝑝−𝑟,𝑑−1 +𝑃1, 𝐿 𝑟 𝑝−𝑟, 𝑑+1 +𝑃1, 𝑚𝑖𝑛 𝑖 𝐿 𝑟 𝑝−𝑟,𝑖 +𝑃2 − 𝑚𝑖𝑛 𝑗 𝐿 𝑟 (𝑝−𝑟,𝑗)

11. Results

12. Depth maps (Cones) Base image Ground truth Pixel-wise SAD 11x11
ASW 11x11
ASW 35x35
SGM

13. Depth maps (Teddy) Base image Ground truth Pixel-wise SAD 11x11
ASW 11x11
ASW 35x35
SGM

14. Depth maps (Tsukuba) Base image Ground truth Pixel-wise SAD 11x11
ASW 11x11
ASW 35x35
SGM

15. Depth maps (Venus) Base image Ground truth Pixel-wise SAD 11x11
ASW 11x11
ASW 35x35
SGM

16. Accuracy

17. Timing

18. Conclusion

19. Pixel-wise Matching
Offers very poor results, for one-to-one, intensity based comparison is not robust against:
Image noise
Illumination changes
Groups of pixels with similar intensities
Etc.
It is very fast, but extremely inaccurate

20. Patch-wise Matching (SAD)
Great improvement of accuracy with respect to Pixel-wise Matching
Problems regarding patch sizing:
Small patches:
The smaller the patch is, the less information we can acquire from neighboring pixels
The Pixel-wise Matching issues are not fully overcome
Big patches:
Disparities discontinuities at object’s edges are not taken into account
Fattening effect
For each match, we have to compare two whole patches
Slower than Pixel-wise Matching, proportionally to the window size (WxH)

21. Patch-wise Matching (ASW)
Weights make possible to have big size patches without remarkable negative effects
Fattening effect disappears and accuracy is improved
Weights have to be calculated for each neighbor
Slower than SAD matching, but generally more accurate and allows big window sizes

22. Semi-Global Matching
Obtaining the base cost from the Pixel-wise Matching makes it very fast
Dynamic programming:
Contributes to the speed of processing
Slightly increases the use of memory
The smoothness constraint grants accuracy
This constraint being multidirectional prevents streaking
As a result, this algorithm is very fast and accurate

23. Bibliography
D. Scharstein, R. Szeliski, R. Zabih, "A Taxonomy and Evaluation of Dense Two- Frame Stereo Correspondence Algorithms“, SMBV’01, 2001.
Kuk-Jin Yoon, In So Kweon, “Adaptive Support-Weight Approach for Correspondence Search”, IEEE transactions on Pattern Analysis and Machine Intelligence, 2006.
Heiko Hirschmüller, “Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information”, CVPR, 2005.