Download presentation

1
**www-video.eecs.berkeley.edu/research**

Introduction to Structured Light (SL) Systems and SL Based Phase Unwrapping R. Garcia & A. Zakhor EECS Department UC Berkeley www-video.eecs.berkeley.edu/research

2
**Outline Background Structured Light Basics Phase Unwrapping**

Our Algorithms Results Summary

3
**Determining Depth of a Scene**

Many applications: Biomedical Industrial Entertainment Navigation Methods for Determining Depth Triangulation Time of flight Depth from (de)focus Other…

4
**Triangulation Based Depth Methods**

Scene observed from multiple views Correspondences between views solved Must know intrinsic and extrinsic parameters for each view Two categories Passive: Stereo – multiple cameras Active: Structured Light – camera with projector or other light source used I J Projector Cam I J Cam [S. Narasimhan]

5
**Traditional Stereo Need 2 or more views of the scene**

Scene texture used to identify correspondences across views Dense stereo remains an active research area in computer vision Algorithms and new Middlebury vision Given rectified images, triangulation is simple Using similar triangles: [Mirmehdi]

6
**Potential Problem with Stereo**

Stereo works well on “textured” images

7
**Potential Problem with Stereo**

Stereo can fail with lack of texture ?

8
**Structured Light (SL) SL places “texture” onto the scene**

Projector patterns identify each scene region Classes of patterns Temporal Spatial Other Survey of patterns [J. Salvi et al., 2004]

9
Temporal Coding Multiple frames are projected to identify scene regions Camera pixel’s intensity change used for correspondence Scene assumed to be static

10
**Spatial Coding Encodes unique information into small regions.**

Fewer captures Less dense reconstruction

11
Spatial Coding Recognize this?

12
**Other Coding Methods Spacetime Coding Viewpoint Coding**

Zhang et al., “Spacetime Stereo: Shape Recovery for Dynamic Scenes”, 2003 Davis et al., “Spacetime Stereo: A Unifying Framework for Depth from Triangulation”, 2003 Viewpoint Coding Young et al., “Viewpoint-Coded Structured Light”, 2007

13
**Structured Light v. Stereo**

Advantages of SL: SL does not require a richly textured scene Solving correspondences is not expensive Each scene point scene receives a unique code Advantages of Stereo: No interference with observed scene Only need acquisitions at one time for capture; SL often needs multiple images Too much texture can be problematic for SL.

14
**Outline Background Structured Light Basics Phase Unwrapping**

Our Algorithms Results Summary

15
**Structured Light Geometry**

Geometry of SL is simple Triangulation performed by ray-plane intersection Intrinsic and extrinsic parameters needed Calibration matrix for camera and projector Transformation between coordinate frame of camera and projector

16
Triangulation Intrinsic and extrinsic parameters Left Cam Projector

17
Triangulation Left Cam Projector

18
**SL for Dynamic Scene Capture**

SL capable of capturing dynamic scenes Want to limit capture time when capturing dynamic scenes “One-shot” approaches require single capture Trade-off Fewer frames -> lower capture resolution More frames -> more sensitive to scene motion

19
**Overview of Phase Shifted Structured Light (SL) Systems**

Project patterns to find correspondences between camera and projector Phase shifted sinusoidal patterns: Fast capture: 3 shots Simple to decode Insensitive to blur Used in optical metrology M periods

20
**Example: Scene Illuminated with Phase-Shifted Sinusoids**

Wrapped Phase I2 I3 Unwrapped Phase M periods in sinusoid Need to unwrap phase

21
**Outline Background Structured Light Basics Phase Unwrapping**

Our Algorithms Results Summary

22
**What is Phase Unwrapping?**

Phase values usually expressed from [-π , π) Would like to recover original continuous phase measurement

23
**Overview of Phase Unwrapping**

Unwrapping assumptions: Single continuous object in scene Slowly varying depth; discontinuities less than |π| 2D phase unwrapping results in relative phase: Need absolute phase for triangulation. Wrapped Phase Unwrapped Phase

24
Creating Point Cloud

25
**Stereo-Assisted Phase Unwrapping**

Stereo assisted phase unwrapping [Wiese 2007]: Results in absolute phase Deals with depth discontinuities System Setup: Two cameras, single projector Cam 1 2 Projector A B C D

26
**Overview of Phase Unwrapping with two Cameras [Weise et al. 2007]**

To resolve the absolute phase for a camera pixel: Determine wrapped phase for all pixels in first and second camera Project a ray from pixel in camera 1 with wrapped phase Project M planes from the projector corresponding to in space Find the M intersections of the M planes with the ray in (1) in 3D space, Project onto camera 2 Compare the M phase values at M pixel locations in camera 2 to the and choose the closest

27
**Stereo Phase Unwrapping [Weise et. al. 2007]**

Right Cam Left Cam Projector

28
**Drawbacks of Stereo Phase Unwrapping**

Must be run twice. Possible to incorrectly assign absolute phase values to corresponding points between views. P A B C D Left Camera Right Camera

29
**Comparison of Stereo Assisted Phase Unwrapping with Merging Stereo and 3D (x,y,t)**

30
**Temporal Inconsistencies**

Consecutive phase images highly correlated Correlated information not used during phase unwrapping Results in inconsistent unwrapping

31
**3D Phase Unwrapping Multi-dimensional phase unwrapping**

2D: traditional image processing 3D: medical imaging (i.e. MRI)

32
**Overview of 3D (x,y,t) Phase Unwrapping**

Treat consecutive phase images as volume of phase values Edges defined between neighboring pixels Quality assigned to each pixel Inversely proportional to spatio-temporal second derivative Want to unwrap according to quality of edges

33
**3D Phase Unwrapping (cont.)**

Evaluate edges in a 3D volume from highest quality to lowest Evaluating edge = unwrapping pixels connected to it Unwrapped pixels connected in chains Grow the chain by adding pixels/edges in x, y, and t Chains merged together

34
**Outline Background Structured Light Basics Phase Unwrapping**

Our Algorithms Results Summary

35
**Consistent Stereo-Assisted Phase-Unwrapping Methods for Structured Light Systems**

36
**Merging Stereo & 3D (x,y,t) Phase Unwrapping**

Goal: Solve for absolute phase offset of a chain by using stereo matching data. Approach: use quality to find phase offset probabilistically M periods in the sinusoidal pattern M possible phase offsets Find offset probability for each pixel in the chain, then combine them to find phase offset for the whole chain How to find phase offset for a pixel P: Use phase difference between wrapped phase at P and its corresponding M projected pixels to generate likelihood of each of the M phase offset values.

37
**Computing Phase Offset for an Entire Chain**

Combine offset probabilities for each pixel in the chain to determine phase offset for the whole chain (a) (b) + π-2δ π-δ P(k1=0) P(k1=1) : P(k1=M-1) P(k2=0) P(k2=1) P(k2=M-1) -π+δ + 0π + 2π Pixels in the same period Pixels in different periods

38
**Comparison with 3D (x,y,t) only**

Proposed 3D (x,y,t) Proposed algorithm avoids unwrapping low quality pixels by using stereo Handling multiple disjoint objects

39
**Comparison with Stereo Only**

Proposed Consecutive unwrapped phase frames and their difference

40
**Comparison of Stereo Assisted Phase Unwrapping with Merging Stereo and 3D (x,y,t)**

41
**Stereo Phase Unwrapping [Weise et. al. 2007]**

Right Cam Left Cam Projector

42
Proposed Method Perform unwrapping w.r.t. projector pixels rather than camera pixels Right Cam Left Cam Projector

43
**Overview of the proposed Method**

For each projector pixel with phase , find the corresponding epipolar line in each image. For each camera, find all points along the epipolar line with phase Find the 3D point in space resulting from intersection of rays resulting from these points with the plane for the projector: N1 points in 3D space for the left camera 1 Ai N2 points in 3D space for the right camera 2 Bi Find the corresponding pair of Ai and Bi closest in 3D space Assign global phase to corresponding pixels of the “best” pair of the two cameras Camera 1 Camera 2

44
**Projector Domain Stereo Method**

Right Cam Left Cam Projector Top down view of the projector plane corresponding to the projector column with absolute phase theta

45
**Finding corresponding “pair” of points in 3D from the two cameras**

Compute pairwise distance for all pairs of points in the two views. The distance between the 3D locations of corresponding points is small. Compute possible correspondences for each projector pixel. Find the correct correspondence labeling for each projector pixel. Use loopy belief propagation (LBP) Distance B1 B2 A1 A2 A3 Possible Labels B1 B2 A1 {A1,B1} {A1,B2} A2 {A2,B1} {A2,B2} A3 {A3,B1} {A3,B2}

46
**Loopy Belief Propagation Cost Function**

Minimize cost function: where: Locations of pixels in Cam A & B Labeling for projector image 3D location of image pixel Set of projector pixels 2D location of image pixel Projector pixel 4- connected pixel neighborhood 3D distance cost threshold 2D distance cost threshold

47
**Local Phase Unwrapping for Remaining Pixels**

Camera Points Computed via LBP Occluded pixels: Corresponding pair are too far apart in 3D space Use quality based local unwrapping Unwrapping order for remaining pixels depends on: Density of stereo unwrapped points Local derivatives Merge pixel density and derivative maps to generate quality map Unwrap from highest to lowest quality. Density Local Derivative Merged

48
**Results Proposed method results in**

Right Camera Proposed Proposed method results in consistent phase results across cameras Left Camera Proposed In a 700 frame sequence, our method: Has same accuracy in 80% of frames Has better than or equal accuracy in 96% of frames. Left Camera [Weise et. al.]

49
**Results: Captured Dynamic Scene**

50
Dynamic Point Cloud

51
**Advantages of Projector Domain Unwrapping**

Only scene points illuminated by the projector can be reconstructed Unwrapping only needs to be performed once for any # of cameras more consistent and efficient. Computational complexity scales with projector resolution rather than image resolution.

52
**Conclusion Provided introduction to the structured light systems**

Presented two phase unwrapping algorithms: A three-dimensional stereo-assisted phase unwrapping method A projector-centric stereo-assisted unwrapping method Results in accurate, consistent phase maps across both views. Results in accurate 3D point clouds

Similar presentations

OK

1 MURI review meeting 09/21/2004 Dynamic Scene Modeling Video and Image Processing Lab University of California, Berkeley Christian Frueh Avideh Zakhor.

1 MURI review meeting 09/21/2004 Dynamic Scene Modeling Video and Image Processing Lab University of California, Berkeley Christian Frueh Avideh Zakhor.

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Download ppt on pulse code modulation math Ppt on centring point Ppt on conservation of water resources Ppt on series and parallel circuits Ppt on council of ministers of ethiopia Ppt on simple carburetor Ppt on sources of energy class 10 download Ppt on sociology of education Ppt on working of stock exchange in india Ppt on earth dam breach