January 29, 2007Videometrics IX, Electronic Imaging 2007, San Jose, CA, U.S.A. Real-Time Range Imaging by Phase-Stamp Method Using Correlation Image Sensor Akira Kimachi 1 and Shigeru Ando 2 1 Osaka Electro-Communication University, Japan 2 The University of Tokyo, Japan

Active Range Imaging Active range imaging — Image sensor + Active light –Time-of-flight methods –Triangulation methods # of frames (scans) Interpixel image processing Image intensity- dependence Light spotManyNoneYes Light stripeManyStripe detectionYes Time stamp light stripe 1NoneYes Structured light1Fringe detection Fringe order Yes Phase-shifting structured light 3-4 or moreNoneNo

Depth Scan angle of sheet beam Stripe image location Light Stripe Range Finder (LSRF)

Time Stamp LSRF (TSRF) Frame-rate operation Pixel-wise sensing –One image for one beam scan –Record a “Time Stamp” for light stripe arrival at each pixel “VLSI silicon range finder” –Gruss et al. (1991) –Kang et al. (1994) –Yokoyama et al. (1994) depth beam angle time stamp

TSRF threshold LSRF Effect of Spatially Nonuniform Reflectance/Background Intensity-dependent detection of light stripe causes errors in LSRF and TSRF nonuniform surface reflectance nonuniform background illumination nonuniform surface reflectance nonuniform background illumination

Objectives Goal — Real-time active range imaging –Frame-rate operation –Pixel-wise sensing (image processing error-free) –Robustness to spatial nonuniformity in Surface reflectance Background illumination Solution — Phase-Stamp Range Finder (PSRF) –Three-Phase Correlation Image Sensor (3PCIS) –Detect the sheet beam angle by “phase stamp”

Three-Phase Time-Domain Correlation Image Sensor (3PCIS) temporal correlation S. Ando and A. Kimachi, IEEE Trans. ED (2003) average intensity

Phase-Stamp Range Finder (PSRF) Three-phase reference signals : Frame rate : Object surface reflectance : Sheet beam intensity : Background light intensity : Arrival time of light stripe Image intensity at

Phase-Stamp Imaging (1) Sheet beam arrives (2) Phase stamped (3) Converted to beam angle one scan one image

Real-Time Range Imaging by PSRF 3PCIS output Phase stamp Total light image Sheet beam image Range imageBeam angle image known function by design Frame-rate range imaging Pixel-wise beam angle detection Robust to spatially nonuniform reflectance/background Phase stamp image uncorrelated to

3PCIS Output Images in PSRF Sheet beam-only intensity imageTotal light intensity image : Surface reflectance : Sheet beam intensity : Background intensity 0   /2  /2 Phase stamp image

Experimental PSRF System 3PCIS camera –200x200 pixels –Frame fps Sheet beam –7 mW laser diode –Cylindrical lens –Scanning scans/s Reference signals –PC ＋ D/A board Hz Camera and geometry calibration –Based on Zhang’s method Reference signals Sheet beam angle

Real-Time 3D Video Capture Phase stampTotal light intensity Sheet beam intensity

3D Imaging Examples left thumb & palm cone-shaped surface 23 mm 17 mm 40 mm

3D Measurement Accuracy Flat mm from the camera phase stamptotal lightsheet beam PhaseStd. dev.4.08 deg Depth Std. dev.2.35 mm Syst. error-1.96 mm 0   /2  /2 3D surface plot

Nonuniform Background Illumination Backgroundabsentstriped Std. dev. [mm] Syst. err. [mm] phase stamptotal lightsheet beam

Nonuniform Surface Reflectance Reflectanceuniformstriped Std. dev. [mm] Syst. err. [mm] phase stamptotal lightsheet beam

Conclusions Phase-stamp range finder using the 3PCIS –Frame-rate operation –Pixel-wise sensing –Robustness to spatial nonuniformity in Surface reflectance Background illumination Evaluation by an experimental system –Real-time range fps –Accuracy Deviation — 2.35 mm → fixed pattern noise (~4 deg) removal Systematic error — mm → geometry calibration

Outline Introduction –Real-time active range imaging Phase-stamp range finder (PSRF) –Three-phase correlation image sensor (3PCIS) –Phase-stamp imaging Experimental system Results Conclusion

Previous Results Other real-time range finders using 3PCIS –Phase-modulated sheet beam Kimachi et al., Trans. IEE Japan (2001) –Phase-modulated structured light Goto et al., 19th Sensor Symposium (2002) Ono et al., 21st Sensor Symposium (2004) PSRF is simpler –No modulated light needed