1. Presenter : Jia-Hao Syu
4/26/2017
Multi-View Reconstruction Preserving Weakly-Supported Surfaces (CVPR 2011) M. Jancosek and T. Pajdla Czech Technical University in Prague
multi-view
Weakly-supported surfaces : low textured walls, windows, cars and ground planes preserve small points in that surface
Czech Technical University
Presenter : Jia-Hao Syu
4/26/2017

2. Motivation 4/26/2017 Original image
3D cloud point (special camera offer depth information of every pixel: introduce later)
[15] : introduce later , that bottle can not reconstruct well
Author’s method
4/26/2017

3. Outline Related Work[15] Weakly-Supported Surfaces Idea
4/26/2017
Outline
Related Work[15]
System diagram
Weakly-Supported Surfaces
Idea
Modified weights
Results
Conclusion
This is outline
4/26/2017

4. 4/26/2017
Related Work
[15] P. Labatut, J. Pons and R.Keriven, “Robust and efficient surface reconstruction from range data”, In Computer Graphics Forum, 2009
2009 計算機圖形論壇
Target
Target : Reconstruct a surface from a set of merged scans (noisy and outliers)
4/26/2017

5. System Diagram 3D cloud points for each cameras
4/26/2017
System Diagram
3D cloud points for each cameras
Combine to one 3D cloud points
Delaunay tetrahedralization of a cloud point
Surface Reconstruction by graph-cut method
3D Delaunay Triangulation
Many image with 3D cloud points
Combine to one 3D cloud points
Delaunay tetrahedralization
Surface Reconstruction by graph cut
Later , I will introduce each system block diagram.
4/26/2017

6. System Diagram 3D cloud points for each cameras
4/26/2017
System Diagram
3D cloud points for each cameras
Combine to one 3D cloud points
Delaunay tetrahedralization of a cloud point
Surface Reconstruction by graph-cut method
3D Delaunay Triangulation
Many image with 3D cloud points
Combine to one 3D cloud points
Delaunay tetrahedralization
Surface Reconstruction by graph cut
Later , I will introduce each system block diagram.
4/26/2017

7. 3D Scanning Technique Contact
4/26/2017
3D Scanning Technique
Contact
Non-Contact
Time-of-flight camera : a range imaging camera system that resolves distance based on the known speed of light
Contact object surface
Non-contact object surface
One method is to use Time of flight camera
A transmitter and a sensor
Compute time for round-trip between A and B
D : distance
c : speed of light
t :  time for round-trip between A and B
4/26/2017

8. System Diagram 3D cloud points for each cameras
4/26/2017
System Diagram
3D cloud points for each cameras
Combine to one 3D cloud points
Delaunay tetrahedralization of a cloud point
Surface Reconstruction by graph-cut method
3D Delaunay Triangulation
Many image with 3D cloud points
Combine to one 3D cloud points
Delaunay tetrahedralization
Surface Reconstruction by graph cut
Later , I will introduce each system block diagram.
4/26/2017

9. 4/26/2017
3D Cloud Points
Acquire a depth map of each camera by 3D scanning technique
Compute depth maps of a 3D cloud points to every camera by plane-sweeping method
We have many depth maps image
Compute a 3D cloud points by plane-sweeping method
4/26/2017

10. Plane-sweeping Method
4/26/2017
Plane-sweeping Method
Pick one pixel P with depth d d
1. Choose one reference image and we pick one pixel p with depth d P
Reference Image
4/26/2017

11. Plane-sweeping Method
4/26/2017
Plane-sweeping Method
Find the nearest n target cameras(ex. n = 4) d P
Reference Image
4/26/2017

12. Plane-sweeping Method
4/26/2017
Plane-sweeping Method
Compute photo consistency by normalized cross-correlation(NCC) d
5*5 windows
Related Camera P
Target Image
Reference Image
4/26/2017

13. Plane-sweeping Method
4/26/2017
Plane-sweeping Method
Normalized Cross-Correlation
The formulation is from wiki
n : the pixel number
f(x,y) : reference image
t(x,y) : target image
The value of NCC is between -1 and 1
 cross-correlation is a measure of similarity of two signal as a function
光源不同 使亮度不均勻 使用Normalized來處理
Set one threshold to preserve the pixel
4/26/2017

14. 4/26/2017
One 3D Cloud Points
Compute photo-consistence between reference and target image
One 3D cloud points can be built
Get all depth-maps of each camera by using a related camera matrix
By the method which is mention before , we can get the one 3D cloud points
Delaunay triangulation is common used method for surface reconstruction
4/26/2017

15. System Diagram 3D cloud points for each cameras
4/26/2017
System Diagram
3D cloud points for each cameras
Combine to one 3D cloud points
Delaunay tetrahedralization of a cloud point
Surface Reconstruction by graph-cut method
3D Delaunay Triangulation
Many image with 3D cloud points
Combine to one 3D cloud points
Delaunay tetrahedralization
Surface Reconstruction by graph cut
Later , I will introduce each system block diagram.
4/26/2017

16. 2D Delaunay Triangulation
Three points can draw a triangle
Add one more point or
4/26/2017

17. 2D Delaunay Triangulation
Draw a circumcircle of triangle or
4/26/2017

18. 2D Delaunay Triangulation
4/26/2017
2D Delaunay Triangulation
Give a set of P point in 2D
4/26/2017

19. 2D Delaunay Triangulation
4/26/2017
2D Delaunay Triangulation
No point in P is inside the circumcircle of any triangle
3D Delaunay triangulation : no point in P is inside the circumsphere of any tetrahedralization
4/26/2017

20. 3D Delaunay triangulation
Example for 3D Delaunay triangulation
4/26/2017

21. System Diagram 3D cloud points for each cameras
4/26/2017
System Diagram
3D cloud points for each cameras
Combine to one 3D cloud points
Delaunay tetrahedralization of a cloud point
Surface Reconstruction by graph-cut method
3D Delaunay Triangulation
Many image with 3D cloud points
Combine to one 3D cloud points
Delaunay tetrahedralization
Surface Reconstruction by graph cut
Later , I will introduce each system block diagram.
4/26/2017

22. Surface Reconstruction
4/26/2017
Surface Reconstruction
We build the 3D Delaunay triangulation
How do you reconstruct surface of the object?
Concept : 3D cloud points are dense near the object surface (cost is small)
S-t graph cut algorithm
Maybe with some noise points, the 3D cloud points are almost dense near the object surface
We want to build the surface by these 3D cloud points and 3D Delaunay triangulation
4/26/2017

23. 4/26/2017
S-t Graph Cut
Source and sink become separated the node of set by a cut
the cost of a cut :
Minimum cut : a cut whose cost is the least over all cuts
We want to use s-t Graph cut but our information are 3D points and Delaunay triangulation
4/26/2017

24. Define Parameters Node : Delaunay tetrahedralization
Edge : triangulation between adjacent tetrahedralizations
s(source) : outside of the surface
t(sink) : inside of the surface P
4/26/2017

25. S-t Graph Cut Algorithm
Perform a Delaunay Triangulation of the 3d point cloud
4/26/2017

26. S-t Graph Cut Algorithm
Add a node P from left tetrahedralization P
4/26/2017

27. S-t Graph Cut Algorithm
Add a node Q from right tetrahedralization P Q
4/26/2017

28. S-t Graph Cut Algorithm
Add two s and t nodes s P Q t
4/26/2017

29. S-t Graph Cut Algorithm10 P Q 3 10 t
4/26/2017

30. S-t Graph Cut Algorithm
This is the surface we want
Outside the surface
inside the surface
4/26/2017

31. Assigned Weight 3D cloud points to camera center(line of sight)
4/26/2017
Assigned Weight
3D cloud points to camera center(line of sight)
Sigma越大 surface可以選的路線越多 smooth效果上升
4/26/2017

32. 4/26/2017

33. Formulation of Cost Function
: Visibility Information from points, cameras
: Quality of reconstructed surface in terms of size of triangles
4/26/2017

34. Weakly Supported Surfaces
Not photo consistent surface : Low-textured walls, windows, cars and ground planes

35. Idea Other information to reconstruct weakly supported surface
4/26/2017
Idea
Other information to reconstruct weakly supported surface
Visual Hull
Camera watch information including algorithm
Union of foreground image
Increasing of number of camera
4/26/2017

36. Idea Define free-support-space
4/26/2017
Idea
Define free-support-space
Highly-supported-free space : union of dense 3D points
Weakly-supported surface with weakly sampled by 3D points are close to the highly-supported-free space
Increasing of camera , the highly-supported-free space increase
4/26/2017

37. Free-space-support T pi r pj Original 3D cloud points
4/26/2017
Free-space-support T pi r pj
Original 3D cloud points
Noise with small alpha vision value because seldom points around the noise point
X : 3D cloud points(before photo-consistence)
4/26/2017

38. Target
Large Jump in Free Space Support as we go from outside to inside.
Next, I give a example of weight assumption

39. Old T-weights

40. Modified Weights
4/26/2017
x = w56-w78

41. Setting up t-weight
4/26/2017

42. System and Spend Time System OS : 64-bit Win7 CPU : Inter Core i7
RAM : 12GB
Dataset
Castle data : 30 images with 3072*2048 resolution
Dragon data : 114 images with 1936*1296 resolution
DataSet/Method
Baseline[CFG 09](mins)
Ours(mins)
Castle 30 32
Dragon 90 94

43. Results
INPUT IMAGE POINT CLOUD
CFG OUR METHOD

44. Results
INPUT IMAGE POINT CLOUD
CGF OUR METHOD

45. Demo Video
Images :
Demo video
4/26/2017

46. Conclusion
Resolve weakly-supported surface by using the information of free-support-space
4/26/2017

47. Reference
M. Jancosek and T. Pajdla, “Multi-View Reconstruction Preserving Weakly-Supported Surface”,  IEEE Conference on Computer Vision and Pattern Recognition, 2011
P. Labatut, J. Pons and R.Keriven, “Robust and efficient surface reconstruction from range data”, In Computer Graphics Forum, 2009
P. Labatut, J. Pons, and R. Keriven., “Efficient Multi-view Reconstruction of Large-Scale Scenes using Interest Points, Delaunay Triangulation and Graph Cuts”, International Conference on Computer Vision, 2007
4/26/2017

48. Reference
M. Jancosek and T. Pajdla , ”Hallucination-free multi-view stereo.”, In RMLE , 2010
M. Jancosek and T. Pajdla, “Removing hallucinations from 3D reconstructions”, Technical Report CMP CTU, 2011
4/26/2017