1. Intelligent Systems Lab. Extrinsic Self Calibration of a Camera and a 3D Laser Range Finder from Natural Scenes Davide Scaramuzza, Ahad Harati, and Roland Siegwar IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, 2007 Presenter: Van-Dung Hoang hvzung@islab.ulsan.ac.kr October 12, 2013

2. Intelligent Systems Lab. 2 Content  Introduction  Camera and LRF model  Bearing angle images  Extrinsic laser-camera calibration  Experiments  Conclusions

3. Intelligent Systems Lab. 3 Introduction  Proposed new method for determining the position and direction of 3D LRF with respect to camera.  This approach does not require and calibration object (e.g. chessboard).  Laser range will be visualize in 3D range and highlighted edge of object.  Correspondence features of camera image and laser image will be manually selected.  Extrinsic parameters will be discovered by using PnP method.

4. Intelligent Systems Lab. 4 Camera model  The camera system consists of perspective camera, catadioptric mirror, which has a single center of projection.  Image point- 3D point estimate (u,v) is point in image. [x, y, z] is a ray from center of camera to point in world. is scalar value. F is a project function, it depend on the camera used.

5. Intelligent Systems Lab. 5 LRF model  3D laser system construct from 2D Laser SICK LMS200.  Combining the rotation of the mirror inside the 2D scanner with the external rotation of the scanner itself.  Impossible to adjust the two centers of rotation exactly on the same point.  These offset values have to be estimated by calibrating the 3D sensor by considering its observation model (other work).  This paper focuses extracting the extrinsic calibration of a 3D laser scanner with a camera is general (does not depend on the sensor model).

6. Intelligent Systems Lab. 6 LRF model  The sensor model can be written:  where ρ ij is the j-th measured distance with orientation θ j in the i-th scan line, and angle  j (external rotation) with the horizontal plane.  (d x, d z ) is offset of the external rotation axis from the center of the laser mirror.

7. Intelligent Systems Lab. 7 Bearing angle images  Highlight depth discontinuities and direction changes in range image so that the user can easily find the corresponding points of the camera image points.  Creation depth image from laser range (a)  Edge detection (b) use Sobel  Only consider the depth between two adjacent points significantly changes.  This work don’t consider the surface direction (normal vectors)  Highlight details of surface along some specific direction (vertical, horizontal,…)

8. Intelligent Systems Lab. 8 Bearing angle images  Bearing Angle (BA) the angle between the laser beam and the segment joining two consecutive measurement points (a) Where ρ i is the i- th depth value in the selected trace of the depth matrix and d  is the corresponding angle increment.  Constructing BA image.

9. Intelligent Systems Lab. 9 Bearing angle images PiPi P i-1 pipi p i-1

10. Intelligent Systems Lab. 10 Bearing angle images

11. Intelligent Systems Lab. 11 Extrinsic laser-camera calibration  Data process:  Collecting data and image.  Computing BA images  Manually select correspondence points between BA image and intensity image.  Store correspondence points where θ C and θ L are unit norm orientation vectors of camera and laser points d L is the point distances in laser frame.

12. Intelligent Systems Lab. 12 Extrinsic laser-camera calibration

13. Intelligent Systems Lab. 13 Extrinsic laser-camera calibration  Algorithm for discovering: rotation and translation parameters.  The extrinsic parameters of transformation between the camera and LRF are determined from known corresponding 3D points. It is solved by using the P3P method

14. Intelligent Systems Lab. 14 Extrinsic calibration extraction

15. Intelligent Systems Lab. 15 Extrinsic calibration extraction  The rotation matrix R =X if det(X)=1, otherwise for failure solution.  Step 4: Translation:

16. Intelligent Systems Lab. 16 Experimental results  Setup the system:  Camera SONY XCD-SX910-CR  Mirror: KAIDAN 360 One VR hyperbolic  Laser SICK LMS 200.  FOV 180 O, resolution 0.5 O  Rotating scanner.  FOV 360 O, resolution 1 O

17. Intelligent Systems Lab. 17 Experimental results Estimation of the translation (meters) versus the number of selected points. Estimation of the rotation (roll, pitch, and yaw angles) versus the number of selected points (the x-axis ranges from 4 to 10).

18. Intelligent Systems Lab. 18 Experimental results  Re-projection laser point onto intensity image

19. Intelligent Systems Lab. 19 Experimental results  Construction 3D point cloud from laser points and vision points.

20. Intelligent Systems Lab. 20 Conclusions  The method uses only a few correspondent points that manually selected by the user from natural scene.  No calibration patterns are required, nor more than one single laser- camera acquisition necessary.  The proposed method relies on a novel technique to visualize the range information obtained from a 3D laser scanner.  The BA images and the application of the method to an omni camera were the two main contributions of the paper.  Proposed approach requires no special equipment and allows the user to calibrate quickly the system.

21. Intelligent Systems Lab. THANK YOU FOR ATTENTION!

22. Intelligent Systems Lab. 22 Project function  This paper the camera coordinate system coincides with the single effective viewpoint.  The x-y plane is orthogonal to the mirror axis. The distance d between focal points of conic and the latus rectum l.