Download presentation

Presentation is loading. Please wait.

Published byChristopher Thompson Modified over 3 years ago

1
Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew Turk 1 1 University of California, Santa Barbara 2 Mitsubishi Electric Research Labs 3 Epson Palo Alto Lab

2
Introduction Correspondence Problem Stereo Near Depth Discontinuities: - Occlusion Problem - Perspective Distortions - Violation of Smoothness Constraints Passive Versus Active Methods

3
Introduction Our Approach: Small Baseline Multi-Flash Illumination - Simple, Inexpensive - Compact, Self-Contained - Discontinuity Preserving

4
Depth Edges with Multi-Flash Raskar, Tan, Feris, Yu, Turk – ACM SIGGRAPH 2004

9
Bottom Flash Top Flash Left Flash Right Flash Ratio images and directions of epipolar traversal Shadow-Free Depth Edges Shadow-FreeDepth Edges

10
Qualitative Depth Map

11
Qualitative Depth Sign of Depth Edge - Indicates which side is the foreground and which side is the background Shadow Width - Encodes object relative distances

12
Sign of Depth Edge + - + - (+) Foreground (-) Background Original Ratio Left Ratio Right Signed Edges

13
Shadow Width Bottom Flash Image Ratio Image Plot Along Scanline

14
Shadow Width Bottom Flash Image Ratio Image Shadow Width Estimation: Meanshift Segmentation algorithm applied on the ratio image

15
Imaging Geometry Object Flash Shadow Camera B z1z1 z2z2 f Shadow Width d

16
Qualitative Depth Working on this Equation … Log Depth Difference Shadow Width Gradient-Domain Problem!

17
Qualitative Depth 1) Compute Sharp Depth Gradient G = (G h,G v ) Log Depth Difference Sign of depth edge 2) Compute Q by integrating G (Poisson Equation) 3) Qualitative depth map Q = exp(Q)

18
Qualitative Depth Useful Prior Information for Stereo !

19
Occlusion Map

20
Partial Occlusion Problem Object Camera Occlusion AB (Seen by A but not by B)

21
Occlusion Bounded by Shadows Object CameraAB Flash Occlusion (Seen by A but not by B)

22
Occlusion Bounded by Shadows Object CameraAB Flash Lower Bound Shadow

23
Occlusion Bounded by Shadows Object CameraAB Flash Upper Bound Shadow

24
Occlusion Bounded by Shadows Object Camera Occlusion AB Average of Upper/Lower Shadow widths Flash

25
Occlusion Bounded by Shadows Occlusion Map Left ViewRight View

26
Discontinuity Preserving Stereo Matching

27
Local Stereo Problem: Shape and size of correlation window - Small Window Ambiguities / Noise - Large Window Problems at Depth Discontinuities Depth Edge Preserving Local Stereo

28
Local Stereo Smooth Disparity Delimited by depth edges + Occlusions Correlation Window Problem: Shape and size of correlation window - Small Window Ambiguities / Noise - Large Window Problems at Depth Discontinuities Depth Edge Preserving Local Stereo

29
Local Stereo Left View Depth Edges + OcclusionGround Truth Challenging Scene: - Ambiguous patterns, textureless regions, geometrically complex object, thin structures

30
Local Stereo Conventional 9x9 Conventional 31x31 Our Approach 31x31 --- Conventional Stereo Our Approach

31
Global Stereo Global Optimization – Markov Random Field (MAP-MRF) X = {x s } Disparity of each pixel (Hidden) Y = {y s } Matching cost at each disparity (Observed) X3X3 X1X1 X2X2 X7X7 X4X4 X6X6 X5X5 X8X8 y1y1 y2y2

32
Global Stereo Global Optimization – Markov Random Field (MAP-MRF) X = {x s } Disparity of each pixel (Hidden) Y = {y s } Matching cost at each disparity (Observed) Data Term Smoothness Term Inference by Belief Propagation [Jian Sun et al, 2003]

33
Global Stereo Qualitative Depth Map as Evidence - Used to set the smoothness term - Information propagation is stopped at depth edges - Encourage disparities for neighboring pixels according to depth difference in qualitative map Occlusion Penalty

34
Global Stereo Conventional Belief Propagation Our Approach RMS: 0.9589 RMS: 0.4590

35
Conclusions Contributions - Stereo with small baseline illumination - Useful Feature Maps (Qualitative Depth + Occlusion Map) - Enhanced Local and Global Stereo Algorithms Pros / Cons - Robust, Simple, Inexpensive and Compact - Limited to handle outdoor scenes and motion Website (datasets, source code) - http://www.cs.ucsb.edu/~rferis/multi-flash-stereo

36
Thank you ! Multi-Flash Stereo Webpage http://www.cs.ucsb.edu/~rferis/multi-flash-stereo Four Eyes Lab, UCSB http://ilab.cs.ucsb.edu

38
Occlusion Bounded by Shadows Occlusion Detection by averaging length of shadows Images taken with light sources surrounding the other camera

Similar presentations

OK

Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.

Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google