1. January 29, 2007Videometrics IX, Electronic Imaging 2007, San Jose, CA, U.S.A. Real-Time Range Imaging by Phase-Stamp Method Using Correlation Image Sensor Akira Kimachi 1 and Shigeru Ando 2 1 Osaka Electro-Communication University, Japan 2 The University of Tokyo, Japan

2. Active Range Imaging Active range imaging — Image sensor + Active light –Time-of-flight methods –Triangulation methods # of frames (scans) Interpixel image processing Image intensity- dependence Light spotManyNoneYes Light stripeManyStripe detectionYes Time stamp light stripe 1NoneYes Structured light1Fringe detection Fringe order Yes Phase-shifting structured light 3-4 or moreNoneNo

3. Depth Scan angle of sheet beam Stripe image location Light Stripe Range Finder (LSRF)

4. Time Stamp LSRF (TSRF) Frame-rate operation Pixel-wise sensing –One image for one beam scan –Record a “Time Stamp” for light stripe arrival at each pixel “VLSI silicon range finder” –Gruss et al. (1991) –Kang et al. (1994) –Yokoyama et al. (1994) depth beam angle time stamp

5. TSRF threshold LSRF Effect of Spatially Nonuniform Reflectance/Background Intensity-dependent detection of light stripe causes errors in LSRF and TSRF nonuniform surface reflectance nonuniform background illumination nonuniform surface reflectance nonuniform background illumination

6. Objectives Goal — Real-time active range imaging –Frame-rate operation –Pixel-wise sensing (image processing error-free) –Robustness to spatial nonuniformity in Surface reflectance Background illumination Solution — Phase-Stamp Range Finder (PSRF) –Three-Phase Correlation Image Sensor (3PCIS) –Detect the sheet beam angle by “phase stamp”

7. Three-Phase Time-Domain Correlation Image Sensor (3PCIS) temporal correlation S. Ando and A. Kimachi, IEEE Trans. ED (2003) average intensity

8. Phase-Stamp Range Finder (PSRF) Three-phase reference signals : Frame rate : Object surface reflectance : Sheet beam intensity : Background light intensity : Arrival time of light stripe Image intensity at

9. Phase-Stamp Imaging (1) Sheet beam arrives (2) Phase stamped (3) Converted to beam angle one scan one image

10. Real-Time Range Imaging by PSRF 3PCIS output Phase stamp Total light image Sheet beam image Range imageBeam angle image known function by design Frame-rate range imaging Pixel-wise beam angle detection Robust to spatially nonuniform reflectance/background Phase stamp image uncorrelated to

11. 3PCIS Output Images in PSRF Sheet beam-only intensity imageTotal light intensity image : Surface reflectance : Sheet beam intensity : Background intensity 0   /2  /2 Phase stamp image

12. Experimental PSRF System 3PCIS camera –200x200 pixels –Frame rate @12.5 fps Sheet beam –7 mW laser diode –Cylindrical lens –Scanning mirror @25 scans/s Reference signals –PC ＋ D/A board –@50 Hz Camera and geometry calibration –Based on Zhang’s method Reference signals Sheet beam angle

13. Real-Time 3D Video Capture Phase stampTotal light intensity Sheet beam intensity

14. 3D Imaging Examples left thumb & palm cone-shaped surface 23 mm 17 mm 40 mm

15. 3D Measurement Accuracy Flat object @540 mm from the camera phase stamptotal lightsheet beam PhaseStd. dev.4.08 deg Depth Std. dev.2.35 mm Syst. error-1.96 mm 0   /2  /2 3D surface plot

16. Nonuniform Background Illumination Backgroundabsentstriped Std. dev. [mm]2.353.11 Syst. err. [mm]-1.96-1.85 phase stamptotal lightsheet beam

17. Nonuniform Surface Reflectance Reflectanceuniformstriped Std. dev. [mm]2.352.39 Syst. err. [mm]-1.96-1.62 phase stamptotal lightsheet beam

18. Conclusions Phase-stamp range finder using the 3PCIS –Frame-rate operation –Pixel-wise sensing –Robustness to spatial nonuniformity in Surface reflectance Background illumination Evaluation by an experimental system –Real-time range imaging @12.5 fps –Accuracy (@540mm) Deviation — 2.35 mm → fixed pattern noise (~4 deg) removal Systematic error — -1.96 mm → geometry calibration

19. Outline Introduction –Real-time active range imaging Phase-stamp range finder (PSRF) –Three-phase correlation image sensor (3PCIS) –Phase-stamp imaging Experimental system Results Conclusion

20. Previous Results Other real-time range finders using 3PCIS –Phase-modulated sheet beam Kimachi et al., Trans. IEE Japan (2001) –Phase-modulated structured light Goto et al., 19th Sensor Symposium (2002) Ono et al., 21st Sensor Symposium (2004) PSRF is simpler –No modulated light needed