Download presentation

Presentation is loading. Please wait.

Published byWilliam Morefield Modified over 2 years ago

1
Multi-view Stereo via Volumetric Graph-cuts George Vogiatzis Roberto Cipolla Cambridge Univ. Engineering Dept. Philip H. S. Torr Department of Computing Oxford Brookes University

2
Multi-view Dense Stereo Calibrated images of Lambertian scene 3D model of scene

3
Multi-view Dense Stereo Volumetric Two main approaches Volumetric Disparity (depth) map

4
Dense Stereo reconstruction problem: Disparity-map Two main approaches Volumetric Disparity (depth) map

5
Shape representation Disparity-maps MRF formulation – good optimisation techniques exist (Graph-cuts, Loopy BP) MRF smoothness is viewpoint dependent Disparity is unique per pixel – only functions represented

6
Shape representation Volumetric – e.g. Level-sets, Space carving etc. Able to cope with non-functions Levelsets: Local optimization Space carving: no simple way to impose surface smoothness

7
Our approach Cast volumetric methods in MRF framework Use approximate surface containing the real scene surface E.g. visual hull Benefits: General surfaces can be represented No depth map merging required Optimisation is tractable (MRF solvers) Smoothness is viewpoint independent

8
Volumetric Graph cuts for segmentation Volume is discretized A binary MRF is defined on the voxels Voxels are labelled as OBJECT and BACKGROUND Labelling cost set by OBJECT / BACKGROUND intensity statistics Compatibility cost set by intensity gradient Boykov and Jolly ICCV 2001

9
Volumetric Graph cuts for stereo Challenges: What do the two labels represent How to define cost of setting them How to deal with occlusion Interactions between distant voxels

10
Volumetric Graph cuts (x) 1. Outer surface 2. Inner surface (at constant offset) 3. Discretize middle volume 4. Assign photoconsistency cost to voxels

11
Volumetric Graph cuts Source Sink

12
Volumetric Graph cuts Source Sink Cost of a cut (x) dS S S cut 3D Surface S [Boykov and Kolmogorov ICCV 2001]

13
Volumetric Graph cuts Source Sink Minimum cut Minimal 3D Surface under photo- consistency metric [Boykov and Kolmogorov ICCV 2001]

14
Photo-consistency Occlusion 1. Get nearest point on outer surface 2. Use outer surface for occlusions 2. Discard occluded views

15
Photo-consistency Occlusion Self occlusion

16
Photo-consistency Occlusion Self occlusion

17
Photo-consistency Occlusion N threshold on angle between normal and viewing direction threshold= ~60

18
Photo-consistency Score Normalised cross correlation Use all remaining cameras pair wise Average all NCC scores

19
Photo-consistency Score Average NCC = C Voxel score = 1 - exp( -tan 2 [ (C-1)/4] / 2 ) 0 1 = 0.05 in all experiments

20
Example

21
Example - Visual Hull

22
Example - Slice

23
Example - Slice with graphcut

24
Example – 3D

25
Protrusion problem Balooning force favouring bigger volumes that fill the visual hull L.D. Cohen and I. Cohen. Finite-element methods for active contour models and balloons for 2-d and 3-d images. PAMI, 15(11):1131– 1147, November 1993.

26
Protrusion problem Balooning force favouring bigger volumes that fill the visual hull L.D. Cohen and I. Cohen. Finite-element methods for active contour models and balloons for 2-d and 3-d images. PAMI, 15(11):1131– 1147, November 1993. (x) dS - dV SV

27
Protrusion problem

29
w ij SOURCE wbwb wbwb Graph h j i w b = h 3 w ij = 4/3 h 2 * ( i + j )/2 [Boykov and Kolmogorov ICCV 2001]

30
Results Model House

31
Results Model House – Visual Hull

32
Results Model House

33
Results Stone carving

34
Results Haniwa

35
Summary Novel formulation for multiview stereo Volumetric scene representation Computationally tractable global optimisation using Graph-cuts. Visual hull for occlusions and geometric constraint Occlusions approximately modelled Questions ?

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google