Efficient Stereo Matching Based on a New Confidence Metric Won-Hee Lee, Yumi Kim, and Jong Beom Ra Department of Electrical Engineering, KAIST, Daejeon, Korea 20th European Signal Processing Conference (EUSIPCO 2012)

Outline Introduction Related Work Proposed Algorithm Experimental Results Conclusion

Introduction

Introduction For the TV application, stereo matching should be performed in real-time. Aggregation kernel size is to be small Aggregation process takes large computation loads May cause problems in a textureless area Texture area information incorrect textureless area Propose a new confidence metric for stereo matching For efficient refinement (with small kernel size) Objective:

Introduction 35X35 5X35 [4] K. Zhang, J. Lu, and G. Lafruit, “Cross-based local stereo matching using orthogonal integral images,” IEEE TCSVT, 2009.

Related Work

Related Work Cross-based stereo matching algorithm[4] Raw matching cost: Aggregated cost: Ud(x) : local support region Ud(x) : the number of pixels in Ud(x)

Related Work Cross-based stereo matching algorithm[4] Winner-take-all: d0(x) : the initial disparity dmax(x) : the maximum disparity

Related Work Confidence metrics[5]: Several metrics were proposed to measure the confidence level of match Utilizing: Aggregated cost Curvature of the cost curve Left-right consistency [5] X. Hu and P. Mordohai, “Evaluation of stereo confidence indoors and outdoors,” in CVPR, 2010

Confidence metrics 1) Matching score metric (MSM) C : aggregated cost di : the disparity that reveals the ith minimum cost White(High confidence) Black(Low confidence)

Confidence metrics 2) Curvature of cost curve metric (CUR)

Confidence metrics 3) Naive peak ratio metric (PKRN)

Confidence metrics 4) Naive winner margin metric (WMNN) computes a margin between two minimum costs normalize it with the sum of total costs

Confidence metrics 5) Left right difference metric (LRD) min{cR(x - d1, dR)}: the minimum value of a cost curve at the corresponding pixel in the right image.

Proposed Algorithm

Framework

Proposed Confidence Metric ‧:Correct estimated pixels X : Incorrect estimated pixels

Proposed Confidence Metric The new metric is proposed as Characteristic: extracts the curvature information across a range larger than that including three cost values in the CUR metric Ud(x) : improve the metric performance for a cost graph with a small curvature. LoG : a Laplacian of Gaussian filter of n-taps

Refinement Weighted median filter Weight 𝒘 𝒊 : 𝒅 𝒊 𝟎 : the initial disparity of neighboring pixels (same color segment) : duplication operator offset a slope of function

Refinement Histogram-based color segmentation algorithm[6]: [6] J. Delon, A. Desolneux, J. L. Lisani, and A. B. Petro, “A nonparametric approach for histogram segmentation,” IEEE TIP, 2007. Refinement Histogram-based color segmentation algorithm[6]:

Refinement The filtering is applied only to the limited number if pixels Due to small size of filtering kernel To enlarge the filtering range Vertically propagate the filtered result of a current pixel Propagation Data After weighted median filtering… Datapropagate = DataA Filtered Disparity A If Weightpropagate > WeightB DisparityB = Disparitypropagate Else Datapropagate = DataB Weight B (current) Color segment index C

Experimental Results

Experimental Results Parameters: Aggregation kernel Filtering kernel T n σ τ 5 x 35 5 x 63 60 7 10 2 n : Laplacian of Gaussian filter of n-taps offset a slope of function

Experimental Results Initial disparity map Bad pixel Confidence map

AUC: Area Under the Curve Experimental Results AUC: Area Under the Curve Venus Tsukuba Teddy Cones

Experimental Results Error rate (Threshold = 1) Cross-based Adaptive support-weight

[4] Experimental Results [10] Proposed

After Aggregation

After Aggregation

After Aggregation

Conclusion

Conclusion Presented an efficient stereo matching algorithm Applying a weighted median filter that is based on the proposed confidence metric. Successfully refine initial disparities. Competitive to the existing algorithms with a large size of aggregation kernel.