1. Stereo Matching Using Dynamic Programming
Jim Rehg
CS 4495/7495 Computer Vision
Lecture 4
Mon Sept 2, 2002

2. Correspondence
It is fundamentally ambiguous, even with stereo constraints
Ordering constraint…
…and its failure

3. Search Over Correspondences
Occluded Pixels
Left scanline
Right scanline
Dis-occluded Pixels
Three cases:
Sequential – cost of match
Occluded – cost of no match
Disoccluded – cost of no match

4. Stereo Matching with Dynamic Programming
Occluded Pixels
Left scanline
Start
Dynamic programming yields the optimal path through grid. This is the best set of matches that satisfy the ordering constraint
Dis-occluded Pixels
Right scanline
End

5. Dynamic Programming 1 1 1 2 2 2 3 3 3
Principle of Optimality for an n-stage assignment problem:

6. Dynamic Programming 1 1 1 2 2 2 3 3 3
Principle of Optimality for an n-stage assignment problem:

7. Dynamic Programming 1 1 1 2 2 2 3 3 3
Principle of Optimality for an n-stage assignment problem:

8. Dynamic Programming 1 1 1 2 2 2 3 3 3
Principle of Optimality for an n-stage assignment problem:

9. Dynamic Programming 1 1 1 2 2 2 3 3 3
Principle of Optimality for an n-stage assignment problem:

10. Dynamic Programming 1 1 1 2 2 2 3 3 3
Back-chaining recovers the optimal path and its cost:

11. Stereo Matching with Dynamic Programming
Occluded Pixels
Left scanline
Scan across grid computing optimal cost for each node given its upper-left neighbors. Backtrack from the terminal to get the optimal path.
Dis-occluded Pixels
Right scanline
Terminal

12. Stereo Matching with Dynamic Programming
Occluded Pixels
Left scanline
Scan across grid computing optimal cost for each node given its upper-left neighbors. Backtrack from the terminal to get the optimal path.
Dis-occluded Pixels
Right scanline
Terminal

13. Stereo Matching with Dynamic Programming
Occluded Pixels
Left scanline
Scan across grid computing optimal cost for each node given its upper-left neighbors. Backtrack from the terminal to get the optimal path.
Dis-occluded Pixels
Right scanline
Terminal

14. Stereo Matching with Dynamic Programming
Occluded Pixels
Left scanline
Scan across grid computing optimal cost for each node given its upper-left neighbors. Backtrack from the terminal to get the optimal path.
Dis-occluded Pixels
Right scanline
Terminal

15. Stereo Matching with Dynamic Programming
Occluded Pixels
Left scanline
Scan across grid computing optimal cost for each node given its upper-left neighbors. Backtrack from the terminal to get the optimal path.
Dis-occluded Pixels
Right scanline
Terminal

16. Stereo Matching with Dynamic Programming
Occluded Pixels
Left scanline
Scan across grid computing optimal cost for each node given its upper-left neighbors. Backtrack from the terminal to get the optimal path.
Dis-occluded Pixels
Right scanline
Terminal

17. Stereo Matching with Dynamic Programming
Occluded Pixels
Left scanline
Scan across grid computing optimal cost for each node given its upper-left neighbors. Backtrack from the terminal to get the optimal path.
Dis-occluded Pixels
Right scanline
Terminal

18. Computing Correspondence
Another approach is to match edges rather than windows of pixels:
Which method is better?
Edges tend to fail in dense texture (outdoors)
Correlation tends to fail in smooth featureless areas

19. Computing Correspondences
Both methods fail for smooth surfaces
There is currently no good solution to the correspondence problem