--- some recent progress Bo Fu University of Kentucky

Paper List 1. Structured Light 3D Scanning in the Presence of Global Illumination. (CVPR 11’) [systematic error introduced by ill lighting condition] Extension: A Practical Approach to 3D Scanning in the Presence of Inter-reflections, Subsurface Scattering and Defocus. (IJCV 12’) 2. Implementing High Resolution Structured Light by Exploiting Projector Blur. (WACV 12’) [increase depth resolution] 3. Vision Processing for Real-time 3D Data Acquisition Based on Coded Structured Light. (IEEE Trans. on Image Processing 2008) [reduce acquisition time]

Structured Light 3D Scanning in the Presence of Global Illumination Mohit Gupta, etc Carnegie Mellon University

Introduction Motivation One important assumption of most structured light techniques does not always hold: scene points receive illumination only directly from the light source.

Introduction Main idea Design patterns that modulate global illumination and prevent the errors at capture time itself. Gray code Min-SWXOR-02

Errors due to Global Illumination Short range effect: sub-surface scattering, defocus Long range effect: inter-reflection, diffusion

1. inter-reflection error

solution

2. sub-surface scattering error

solution

Error formulation

α: projector defocus fraction

Error formulation Correct binarization: Note: without global illumination (L g =0), defocus (α = 1), this condition automatically holds

Error formulation Long range effect diffuse and specular inter-reflection Consequence: low frequency decode error. Since the low frequency pattern correspond to the higher-order bits, this results in a large error in the recovered shape.

Error formulation Short-range effect sub-surface scattering and defocus Consequence: loss of depth resolution

Pattern for error prevention How to design pattern to prevent both long range effect and short range effect? pattern with only high frequencies can prevent long range effect. pattern with only low frequencies can prevent short range effect. It is possible to design code with only high frequency patterns, while for short range effect, patterns with large minimum stripe-width can be designed.

Pattern for error prevention

1. For long range effect Base pattern Gray code XOR

Pattern for error prevention 1. For long range effect

Pattern for error prevention 2. For short range effect Design codes with large minimum stripe-width (min-SW) well studied in combinatorial mathematics [1]. [1] binary gray codes with long bit runs. The electronic journal of combinatorics. 2003

Pattern for error prevention 2. Ensemble of codes for general scenes using four codes optimized for different effects: XOR-04 and XOR-02 for long range effect Gray code with maximum min-SW and Gray code for short range effect Rule: if any two agree within a small threshold, that value is returned as true depth,.

Experiment Please refer to the paper (IJCV preferred)

Implementing High Resolution Structured Light by Exploiting Projector Blur CamilloTaylor University of Pennsylvania

Introduction Motivation 1. With standard structured light decoding schemes, one is limited by the resolution of the projector. The quantization of the projector ultimately limits the accuracy of the reconstruction.

Introduction Motivation 2. Growing disparity between the resolution of the image sensor and the resolution of the projector systems. 1600* *480 VS

Introduction Main idea By exploit the blur induced by the optics of the projector, subpixel correspondences between camera frame and projector frame can be established Major comparison: Li Zhang, Shree Nayar. Projection Defocus Analysis for Scene Capture and Image Display. (TOG 06’)

Approach The effective irradiance that a pixel in the projector contributes to a point in the scene is related to the displacement between the projection of that scene point on the projector frame and the center of the pixel. This falloff is modeled as a Gaussian I: observed scene radiance measured at a pixel in the camera f: BRDF at the scene point E: irradiance supplied to the corresponding scene point by the projector

Approach k: stripe index σ: width of the blur kernel at point in the scene δ: projection of scene point in the projector frame (with subpixel precision) k δ

Approach I0: scene irradiance due to ambient illumination (known) δ : floating point offset between -0.5 and 7.5

Experiment For more result, please refer to original paper

Vision Processing for Real-time 3D Data Acquisition Based on Coded Structured Light S. Y. Chen, etc City University of Hong Kong

Motivation Conventional structured light system can not be applied to moving surfaces since multiple patterns must be projected.

Main idea Grid solid pattern for 3d reconstruction with fast matching strategies.

Thank you