1. Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard CVPR 2014
Yongho Shin

2. 𝑂(𝑊𝐻𝑁𝐷) 𝑂( 2 10 𝑊𝐻𝑁𝐷) Problems
High-resolution images require long time for computing a disparity map
Complexity for general local methods for 2x size images
𝑂(𝑊𝐻𝑁𝐷)
𝑂( 2 10 𝑊𝐻𝑁𝐷)
This is a problem of this paper. If we use the high resolution image, it requires a lot of time for calculating disparity map. For example, we have image scaled up 4 times, and we needs almost 1000 times time. x4

3. Related works 𝑂(𝑊𝐻𝐷) Semi-global matching
Optimize following energy function
𝐸=𝐸 𝑑𝑎𝑡𝑎 +𝐸( 𝐷 𝑝 − 𝐷 𝑞 =1)+𝐸( 𝐷 𝑝 − 𝐷 𝑞 >1)
NP-hard problem!!
Approximate methods operate in adequate computing time, but still slow
Dynamic programming gives faster way, but erroneous result
Instead do dynamic programming along many directions
It cannot model second-order smoothness
𝑂(𝑊𝐻𝐷)
For efficiently capturing disparity map, a lot of athors give a good manners. One of the good method is a SGM. SGM is the optimization technique. It minimize following energy function. But this energy function is NP –hard problem.

4. Related works Efficient large-scale stereo matching
Stereo matching based on search space reduction
Computation GCPs
Delaunay triangulation on GCPs
Matching on triangles with restricted range

5. Segment-Based Stereo Matching Using Belief Propagation
Very related work

6. Matching with a segmentation
Initial matching
Any matching method can be used
Initial matching
Extraction of
reliable pixels
model parameter
from each segment
Assignment of
optimal parameter
for each segment by BP
Noisy result

7. Matching with a segmentation
Extraction of reliable pixels
Simple cross checking method is used
Occlusion region can be detected
Initial matching
Extraction of
reliable pixels
model parameter
from each segment
Assignment of
optimal parameter
for each segment by BP
Left image
Right image
Left result
Right result

8. Matching with a segmentation
Extraction of model parameter from each segment
At each segment, a model parameter is extracted using reliable pixels and robust statistical technique
Add the parameter to a parameter set
Initial matching
Extraction of
reliable pixels
model parameter
from each segment
Assignment of
optimal parameter
for each segment by BP
Reliable pixels
Segments

9. Matching with a segmentation
Extraction of model parameter from each segment
At each segment, a model parameter is extracted using reliable pixels and robust statistical technique
Add the parameter to a parameter set
Initial matching
Extraction of
reliable pixels
model parameter
from each segment
Assignment of
optimal parameter
for each segment by BP
Parameter
Parameter Set

10. Matching with a segmentation
Assignment of optimal parameter for each segment by BP
Assign an optimal parameter for each segment as total energy can be minimized
Initial matching
Extraction of
reliable pixels
model parameter
from each segment
Assignment of
optimal parameter
for each segment by BP
Parameter #29
Parameter #29
Parameter Set

11. Matching with a segmentationb c d
a : Initial disparity map
b : Interpolated result
c : Reliable pixel map
d : Result from
a segmentation

12. Matching with a segmentation
What they did
Make plane parameter by segment and initial disparity map
Find optimal plane parameters for each segment of the image
Select optimal parameters by BP

13. Proposed method

14. Information for understanding
What they do
Make plane parameter by feature points
Find optimal plane parameters for each tiles of the image
Allowing objects having curved surface
Select optimal parameters by SGM

15. Hypothesis generation
Proposed method

16. Hypothesis generation
Feature matching
By Harris corner keypoints and upright DAISY descriptors
Matching only points along near epipolar line
Due to stereo matching
But, they allow small vertical misalignments
First round
Initial set of matches are selected using the ratio test heuristic
Second round
For obtaining more matched features
Horizontal search range is reduced using local estimates

17. Hypothesis generation
Vertical alignment
Correct for small vertical misalignments from errors in rectification
By fitting a global linear model using RANSAC with matched features
𝑑 𝑦 =𝑎𝑦+𝑏

18. Hypothesis generation
Disparity plane estimation
Cluster matched points and find plane parameters
Find k number of planes
Using variational approach used for mesh simplification
Graph based approach with priority queue

19. Local plane sweeps
Proposed method

20. Local plane sweep Plane for tiles having parallax
Because there are curved objects in the world
Hence, gives range of ±T pixels of parallax from plane
For each plane, investigate similarity among range 2T
Optimize by SGM

21. Local plane sweep Identifying in-range disparities
By disparity map, they give cost U AD
NCC
JUMP

22. Proposal generation
Proposed method

23. Proposal generation Initial proposals Online proposals
Find the planes with associated points within each tile
Online proposals
Find frequent plane parameter for each tile
Propagate the parameter to neighbors

24. Global optimization
Proposed method

25. Power SGM!! Global optimization We have Plane parameters for each tile
Cost U
Energy function
Power SGM!!

26. Experiments

27. Quantitative results

28. Qualitative results