Introduction Stereo disparity estimation is an integral problem associated with 3D content delivery Two type of existing algorithm – Local – Global : minimizing energy function Even applying the best of existing methods to individual frames of stereo sequences yields temporally inconsistent disparity maps (fast, lack the accuracy) (slow)
Introduction The goal is to present a method to generate accurate and spatio-temporally consistent disparity maps from complex stereo video sequences.
Proposed Method Use Image-based technique – Video disparity problem in space-time is computationally impractical – But we lose the consistency between consecutive frames noisy Improve the temporal consistency
Proposed Method Image-based disparity map estimation In this step, disparity maps are computed for each frame individually. Use a global method using Hierarchical Belief Propagation (HBP) for inferencing. Energy function : P : set of pixels in an image L : finite set of labels A labeling f assigns a label fp ϵ L to each pixel p ϵ P Data costDiscontinuity cost How well the labeling fit the node
Proposed Method Image-based disparity map estimation Discontinuity cost enforces the assumption that labels should vary slowly. Except for significant changes along object boundaries
Proposed Method Image-based disparity map estimation Data cost is computed over a large window for each pixel using locally adaptive support weights   K. J. Yoon and I. S. Kweon, “Locally Adaptive Support-Weight Approach for Visual Correspondence Search,” in CVPR, 2005. Strength of grouping by similarity Strength of grouping by proximity : Color difference : Spatial distance only points with a high probability of belonging to the same object contribute significantly to the cost calculation
Proposed Method Image-based disparity map estimation use the method of  to minimize the energy over the entire image in a coarse-to-ﬁne manner. Use the hierarchy to reduce the number of message passing iterations.  P. Felzenszwalb and D. Huttenlocher, “Efﬁcient Belief Propagation for Early Vision,” in CVPR, 2004, pp. 261–268.
Proposed Method temporal consistency Disparity should be a piecewise smooth function in time, except for discontinuities at object borders Consider the sequence of disparity maps as a space-time volume x y t
Proposed Method temporal consistency TV-norm Add (β x, β y, β t ) so that we can control the relative emphasis Total variation Forward difference  S. H. Chan, R. Khoshabeh, K. B. Gibson, P. E. Gill, and T. Q. Nguyen,“An augmented lagrangian method for total variation video restoration,” in ICASSP, May 2011 allows us to handle both spatial and temporal consistency simultaneously, by tuning the parameters (βx, βy, βt)