Presentation is loading. Please wait.

Presentation is loading. Please wait.

Computer Vision and Robotics Group Institut d’Informàtica i Aplicacions Jordi Pagès Joaquim Salvi Overview of coded light projection techniques for automatic.

Similar presentations


Presentation on theme: "Computer Vision and Robotics Group Institut d’Informàtica i Aplicacions Jordi Pagès Joaquim Salvi Overview of coded light projection techniques for automatic."— Presentation transcript:

1 Computer Vision and Robotics Group Institut d’Informàtica i Aplicacions Jordi Pagès Joaquim Salvi Overview of coded light projection techniques for automatic 3D profiling

2 2 Formació Presentation outline 1/36 Introduction Coded pattern classification –Time-multiplexing –Spatial codification –Direct codification Experimental results Sub-pixel matching Conclusions & guidelines Conclusions Sub-pixel matching Experiments Classification Introduction

3 3 Conclusions Sub-pixel matching Experiments Classification Introduction Shape acquisition Stereovision Encoding/ decoding Introduction: shape acquisition 2/36 Shape acquisition techniques Contact Non-destructiveDestructive CMMJointed arms Slicing Non-contact Reflective Transmissive Non-optical Optical Industrial CT Microwave radarSonar Passive Active StereoShadingSilhouettesTextureMotion Shape from X Imaging radar Triangulation Interferometry (Coded) Structured light MoireHolography Stereo Source: Brian Curless

4 4 Conclusions Sub-pixel matching Experiments Classification Introduction Shape acquisition Stereovision Encoding/ decoding Introduction: passive stereovision 3/36 Correspondence problem geometric constraints  search along epipolar lines 3D reconstruction of matched pairs by triangulation

5 5 Conclusions Sub-pixel matching Experiments Classification Introduction Shape acquisition Stereovision Encoding/ decoding Introduction: active stereo (coded structured light) 4/36 One of the cameras is replaced by a light emitter Correspondence problem is solved by searching the pattern in the camera image (pattern decoding)

6 6 Conclusions Sub-pixel matching Experiments Classification Introduction Shape acquisition Stereovision Encoding/ decoding Introduction: pattern encoding/decoding (I) 5/36 A pattern is encoded when after projecting it onto a surface, a set of regions of the observed projection can be easily matched with the original pattern. Example: pattern with two-encoded-columns The process of matching an image region with its corresponding pattern region is known as pattern decoding  similar to searching correspondences Decoding a projected pattern allows a large set of correspondences to be easily found thanks to the a priori knowledge of the light pattern Pixels in red and yellow are directly matched with the pattern columns Codification using colors Object scene

7 7 Conclusions Sub-pixel matching Experiments Classification Introduction Shape acquisition Stereovision Encoding/ decoding 6/36 Introduction: pattern encoding/decoding (II) Two ways of encoding the correspondences: single and double axis codification  it determines how the triangulation is calculated Decoding the pattern means locating points in the camera image whose corresponding point in the projector pattern is a priori known Double-axis encoding Triangulation by line-to-plane intersection Triangulation by line-to-line intersection Single-axis encoding

8 8 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification Coded structured light patterns: a classification proposal 7/36

9 9 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification Time-multiplexing 8/36 The time-multiplexing paradigm consists of projecting a series of light patterns so that every encoded point is identified with the sequence of intensities that receives The most common structure of the patterns is a sequence of stripes increasing its length by the time  single-axis encoding Advantages: –high resolution  a lot of 3D points –High accuracy (order of  m) –Robustness against colorful objects since binary patterns can be used Drawbacks: –Static objects only –Large number of patterns Pattern 1 Pattern 2 Pattern 3 Projected over time Example: 3 binary- encoded patterns which allows the measuring surface to be divided in 8 sub-regions

10 10 Formació Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification Time-multiplexing: binary codes (I) 9/36 Coding redundancy: every edge between adjacent stripes can be decoded by the sequence at its left or at its right

11 11 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification Time-multiplexing: Binary codes (II) 10/36 Pattern 1 Pattern 2 Pattern 3 Projected over time Every encoded point is identified by the sequence of intensities that receives n patterns must be projected in order to encode 2 n stripes Example: 7 binary patterns proposed by Posdamer & Altschuler … Codeword of this píxel:  identifies the corresponding pattern stripe

12 12 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification Time-multiplexing: n-ary codes (I) 11/36 3 patterns based on a n-ary code of 4 grey levels (Horn & Kiryati)  64 encoded stripes n-ary codes reduce the number of patterns by increasing the number of projected intensities (grey levels/colours)  increases the basis of the code The number of patterns, the number of grey levels or colours and the number of encoded stripes are strongly related  fixing two of these parameters the reamaining one is obtained Using a binary code, 6 patterns are necessary necessary to encode 64 stripes

13 13 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification Time-multiplexing: n-ary codes (II) 12/36 n-ary codes reduce the number of patterns by increasing the number of projected intensities (grey levels/colours)

14 14 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification Time-multiplexing: Gray code + Phase shifting (I) 13/36 Gühring’s line-shift technique A sequence of binary patterns (Gray encoded) are projected in order to divide the object in regions An additional periodical pattern is projected The periodical pattern is projected several times by shifting it in one direction in order to increase the resolution of the system  similar to a laser scanner Example: three binary patterns divide the object in 8 regions Without the binary patterns we would not be able to distinguish among all the projected slits Every slit always falls in the same region

15 15 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification 14/36 A periodical pattern is shifted and projected several times in order to increase the resolution of the measurements The Gray encoded patterns permit to differentiate among all the periods of the shifted pattern Time-multiplexing: Gray code + Phase shifting (II)

16 16 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification Formació Time-multiplexing: hybrid methods (I) 15/36 In order to decode an illuminated point it is necessary to observe not only the sequence of intensities received by such a point but also the intensities of few (normally 2) adjacent points The number of projected patterns reduces thanks to the spatial information that is taken into account The redundancy on the binary codification is eliminated Hall-Holt and Rusinkiewicz technique: 4 patterns with 111 binary stripes Edges encoding: 4x2 bits (every adjacent stripe is a bit) Pattern 1 Pattern 2 Pattern 3 Pattern 4 Edge codeword:

17 17 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification 16/36 Time-multiplexing: hybrid methods (II)

18 18 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification Spatial Codification 17/36 Spatial codification paradigm encodes a set of points with the information contained in a neighborhood (called window) around them The codification is condensed in a unique pattern instead of multiplexing it along time The size of the neighborhood (window size) is proportional to the number of encoded points and inversely proportional to the number of used colors The aim of these techniques is to obtain a one-shot measurement system  moving objects can be measured Advantages: ― Moving objects supported ― Possibility to condense the codification to a unique pattern Drawbacks: ― Discontinuities on the object surface can produce erroneous window decoding (occlusions problem) ― The higher the number of used colours, the more difficult to correctly identify them when measuring non- neutral surfaces ― Maximum resolution cannot be reached

19 19 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification Spatial codification: non-formal codification (I) 18/36 The first group of techniques that appeared used codification schemes with no mathematical basis Drawbacks: - the codification is not optimal and often produces ambiguities since different regions of the pattern are identical - the structure of the pattern is too complex for a good image processing Maruyama and Abe  complex structure based on slits containing randomly placed cuts Durdle et al.  periodic pattern

20 20 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification Spatial codification: non-formal codification (II) 19/36

21 21 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification 20/36 Spatial codification: De Bruijn sequences (I) A De Bruijn sequence (or pseudorrandom sequence) of order m over an alphabet of n symbols is a circular string of length n m that contains every substring of length m exactly once (in this case the windows are unidimensional) The De Bruijn sequences are used to define coloured slit patterns (single axis codification) or grid patterns (double axis codification) In order to decode a certain slit it is only necessary to identify one of the windows in which it belongs to Zhang et al.: 125 slits encoded with a De Bruijn sequence of 8 colors and window size of 3 slits Salvi et al.: grid of 29  29 where a De Bruijn sequence of 3 colors and window size of 3 slits is used to encode the vertical and horizontal slits m=4 (window size) n=2 (alphabet symbols)

22 22 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification Spatial codification: De Bruijn sequences (II) 21/36

23 23 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification Spatial codification: M-arrays (I) 22/36 An m-array is the bidimensional extension of a De Bruijn sequence. Every window of w  h units appears only once. The window size is related with the size of the m-array and the number of symbols used Morano et al. M-arry represented with an array of coloured dots Example: binary m- array of size 4  6 and window size of 2  2 M-array proposed by Vuylsteke et al. Represented with shape primitives Shape primitives used to represent every symbol of the alphabet

24 24 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification 23/36 Spatial codification: M-arrays (II)

25 25 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification Direct Codification 24/36 Every encoded pixel is identified by its own intensity/colour Since the codification is usually condensed in a unique pattern, the spectrum of intensities/colours used is very large Additional reference patterns must be projected in order to differentiate among all the projected intensities/colours: Ambient lighting (black pattern) Full illuminated (white pattern) … Advantages: –Reduced number of patterns –High resolution can be teorically achieved Drawbacks: –Very noisy in front of reflective properties of the objects, non-linearities in the camera spectral response and projector spectrum  non-standard light emitters are required in order to project single wave-lengths –Low accuracy (order of 1 mm)

26 26 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification 25/36 Direct codification: grey levels (I) Every encoded point of the pattern is identified by its intensity level Carrihill and Hummel Intensity Ratio Sensor: fade from balck to white Requirements to obtain high resolution Every slit is identified by its own intensity Every slit must be projected using a single wave-length Cameras with large depth-per-pixel (about 11 bits) must be used in order to differentiate all the projected intensities

27 27 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification Direct codification: grey levels (II) 26/36

28 28 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification Direct codification: Colour (I) 27/36 Every encoded point of the pattern is identified by its colour Tajima and Iwakawa rainbow pattern (the rainbow is generated with a source of white light passing through a crystal prism) T. Sato patterns capable of cancelling the object colour by projecting three shifted patterns (it can be implemented with an LCD projector if few colours are projected  drawback: the pattern becomes periodic in order to maintain a good resolution)

29 29 Conclusions Sub-pixel matching Experiments Introduction Classification Time- multiplexing Spatial codification Direct codification 28/36 Direct codification: Colour (II)

30 30 Conclusions Sub-pixel matching Classification Introduction Experiments Quantitative results Qualitative results Experiments: quantitative results 29/36 Technique StDev(  m) 3D Points Resolution (%)* Patterns Time- multiplexing Binary code Posdamer (128 stripes) n-ary codeHorn (64 stripes) Phase Shift + Gray code Gühring (113 slits) Spatical codification De Bruijn sequence De Bruijn (64 slits) Salvi (29  29 grid) M-array Morano (45  45 array) Direct codification ColourSato (64 stripes) Results obtained by reconstructing 30 times two flat panels separated by 40mm. The distance between both panels obtained for each technique was calculated for every reconstruction. The Standard Deviation is indicated. (*) % of pixels inside a window of 515x226 of the camera image that have been triangulated

31 31 Conclusions Sub-pixel matching Introduction Experiments Quantitative results Qualitative results Experiments: qualitative results 30/36 Time- multiplexing Posdamer (128 stripes)Horn (64 stripes)Gühring (113 slits) Spatial codification De Bruijn (64 slits)Salvi (29x29 slits)Morano (45x45 dot array) Direct codification Sato (64 slits) Gühring De Bruijn Classification

32 32 Conclusions Experiments Introduction Sub-pixel matching Stripe patterns Other patterns Classification Sub-pixel matching: a key point 31/36 A key point in the accuracy of the 3D measurements is to locate the correspondences between camera image and projector pattern with sub-pixel coordinates Qualitative example: Horse reconstruction with pixel-accuracy triangulation (20000 points) Horse reconstruction with subpixel-accuracy triangulation (10000 points) Camera image The reflected rays of light fall between ajdacent pixels in the camera sensor This must be taken into account when triangulating the 3D points Projected pattern

33 33 Conclusions Experiments Sub-pixel matching Stripe patterns Other patterns Classification Sub-pixel matching: stripe-patterns 32/36 Only points belonging to the edges between adjacent stripes are decoded and reconstructed. Two possible strategies: - Intersecting the stripe intensity profile with and adaptative binarization threshold (calculated from two additional images: full illuminated and ambient lighting) - More accurate: Projecting positive and negative patterns and intersecting the stripe profiles + Introduction

34 34 Conclusions Experiments Introduction Sub-pixel matching Stripe patterns Other patterns Classification Sub-pixel matching: other patterns 33/36 Arry of dots: the subpixel position of the dots is often calculated with their mass centre  not very accurate since the circles are observed like deformed ellipses due to the change of perspective and the measuring surface Slit patterns: every observed slit can be modelled with a gaussian profile and peak detectors can be applied (like Blais & Rioux detector)  very accurate

35 35 Sub-pixel matching Experiments Introduction Conclusions Guidelines Classification Conclusions 34/36 Types of techniques  Time-multiplexing Highest resolution High accuracy Easy implementation Inapplicability to moving objects Large number of patterns Spatial codification Can measure moving objects A unique pattern is required Lower resolution than time- multiplexing More complex decoding stage Occlusions problem Direct codification High resolution Few patterns Very sensitive to image noise  cameras with large depth-per-pixel required Sensitive to limited bandwith of LCD projectors  special projector devices are usually required Inapplicability to moving objects

36 36 Sub-pixel matching Experiments Introduction Conclusions Guidelines Classification Guidelines 35/36 RequirementsBest technique High accuracy Highest resolution Static objects No matter the number of patterns Phase shift + Gray code  Gühring’s line-shift technique High accuracy High resolution Static objects Minimum number of patterns N-ary pattern  Horn & Kiryati Caspi et al. High accuracy Good resolution Moving objects De Bruijn pattern  Zhang et al.

37 Computer Vision and Robotics Group Institut d’Informàtica i Aplicacions Jordi Pagès Joaquim Salvi END


Download ppt "Computer Vision and Robotics Group Institut d’Informàtica i Aplicacions Jordi Pagès Joaquim Salvi Overview of coded light projection techniques for automatic."

Similar presentations


Ads by Google