Aleixo Cambeiro Barreiro 광주과학기술원 컴퓨터 비전 연구실 Stereo Matching Aleixo Cambeiro Barreiro 광주과학기술원 컴퓨터 비전 연구실

Index Abstract Algorithms used Results Conclusion Bibliography

Abstract

Stereo Matching Process of generation of a depth map for one scene having several pictures of it Achieved by comparing the disparities of the objects’ positions between pixels Takes into account cameras’ conic fields of vision

Goals In this experiment we intend to compare several dense stereo matching algorithms To this end, we have considered: Time elapsed during computation Results accuracy

Algorithms used

Pixel-wise Matching Intensity, Winner Takes All pixel-to-pixel comparison Base image Target image O  Matching pixel  Matching candidates *  Current candidate

Patch-wise Matching (I) Similar to pixel-wise matching Also takes into account neighbors of the matching pixels to improve accuracy Base image Target image O  Matching pixel  Matching candidates *  Current candidate x  Current neighbor

Patch-wise Matching (II) It can be subdivided according to the way the patches are compared We have tested the following two comparison methods: Sum of Absolute Differences (SAD): Sums differences between corresponding pixels in both patches This absolute differences sum is used as matching cost Doesn’t take into account depth differences between pixels in the same patch Adaptive Support-Weight: Similar to SAD, but weights are added to intensity differences to take into account depth disparities between pixels in the same patch, according to next equation: 𝑤 𝑝,𝑞 = exp − ∆ 𝑐 𝑝𝑞 𝛾 𝑐 + ∆ 𝑔 𝑝𝑞 𝛾 𝑝

Semi-Global Matching (SGM) Uses pixel-wise matching as a base cost Adds an one-dimensional, multidirectional smoothness constraint This generates an energy function that should be minimized The minimization can be efficiently achieved through dynamic programming Cost is propagated according to the next equation: 𝐿 𝑟 𝑝,𝑑 =𝐶 𝑝,𝑑 + min 𝐿 𝑟 𝑝−𝑟,𝑑 , 𝐿 𝑟 𝑝−𝑟,𝑑−1 +𝑃1, 𝐿 𝑟 𝑝−𝑟, 𝑑+1 +𝑃1, 𝑚𝑖𝑛 𝑖 𝐿 𝑟 𝑝−𝑟,𝑖 +𝑃2 − 𝑚𝑖𝑛 𝑗 𝐿 𝑟 (𝑝−𝑟,𝑗)

Results

Depth maps (Cones) Base image Ground truth Pixel-wise SAD 11x11 ASW 11x11 ASW 35x35 SGM

Depth maps (Teddy) Base image Ground truth Pixel-wise SAD 11x11 ASW 11x11 ASW 35x35 SGM

Depth maps (Tsukuba) Base image Ground truth Pixel-wise SAD 11x11 ASW 11x11 ASW 35x35 SGM

Depth maps (Venus) Base image Ground truth Pixel-wise SAD 11x11 ASW 11x11 ASW 35x35 SGM

Accuracy

Timing

Conclusion

Pixel-wise Matching Offers very poor results, for one-to-one, intensity based comparison is not robust against: Image noise Illumination changes Groups of pixels with similar intensities Etc. It is very fast, but extremely inaccurate

Patch-wise Matching (SAD) Great improvement of accuracy with respect to Pixel-wise Matching Problems regarding patch sizing: Small patches: The smaller the patch is, the less information we can acquire from neighboring pixels The Pixel-wise Matching issues are not fully overcome Big patches: Disparities discontinuities at object’s edges are not taken into account Fattening effect For each match, we have to compare two whole patches Slower than Pixel-wise Matching, proportionally to the window size (WxH)

Patch-wise Matching (ASW) Weights make possible to have big size patches without remarkable negative effects Fattening effect disappears and accuracy is improved Weights have to be calculated for each neighbor Slower than SAD matching, but generally more accurate and allows big window sizes

Semi-Global Matching Obtaining the base cost from the Pixel-wise Matching makes it very fast Dynamic programming: Contributes to the speed of processing Slightly increases the use of memory The smoothness constraint grants accuracy This constraint being multidirectional prevents streaking As a result, this algorithm is very fast and accurate

Bibliography D. Scharstein, R. Szeliski, R. Zabih, "A Taxonomy and Evaluation of Dense Two- Frame Stereo Correspondence Algorithms“, SMBV’01, 2001. Kuk-Jin Yoon, In So Kweon, “Adaptive Support-Weight Approach for Correspondence Search”, IEEE transactions on Pattern Analysis and Machine Intelligence, 2006. Heiko Hirschmüller, “Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information”, CVPR, 2005.