1. 3D M otion D etermination U sing µ IMU A nd V isual T racking 14 May 2010 Centre for Micro and Nano Systems The Chinese University of Hong Kong Supervised by Prof. Li Lam Kin Kwok, Mark

2. Outline Brief summary of previous works Detail of Visual Tracking System (VTS) -Perspective Camera Model -Procedure of Pose Estimation Current Results of VTS Conclusion Future Plan

3. Previous Works Implement Harris Corner Finding Algorithm -Automatic finding good features Improve the performance of LK Tracking Method -Reduce the noise generated by inconstant lighting Find some information about high speed camera (>60fps)

4. Previous Works

5. Detail of Visual Tracking System Select ROI from captured image Extract Good Features (Harris Algorithm) Motion Tracking (LK Tracking Method) Pose Estimation Position and Orientation (Camera Coordinate) Coordinate Transformation Final Pose of Camera (World Coordinate)

6. Perspective Camera Model Optical Axis Square Grid Image Plan f lili P1P1 P3P3 P2P2 P4P4 p1p1 p2p2 p3p3 p4p4 C C: Optical Center f : Focal Length l i : Distance between 3D feature points and the optical center P i : 3D Feature Points on the square grid p i : Corresponding 2D projected image points cycy cxcx czcz { C } wywy wxwx wzwz { W } { W } : World Coordinate { I } : Image Coordinate { C } : Camera Coordinate v u { I }

7. Perspective Camera Model Relationship between image point and 3D scene point Image Plan Optical Axis Scene Optical Center f cZcZ cxcx cXcX z x { C }

8. Pose Estimation Procedure Calibrate camera (obtain interior parameter) Target Dimension Target Image Step 1: Calibration and Measurement Calculate distance between target and camera Step 2: Recover Pose of Camera ( Respect to Camera Coordinate) Calculate Transformation Matrix Step 3: Recover Transformation Matrix between Camera to World Coordinate Final Pose Step 4: Transform the coordinate to World Coordinate

9. Pose Estimation (Step 1) Using square pattern (with known dimensions) to calibrate a camera

10. Pose Estimation (Step 2) Image to Camera Coordinate Transformation Image Coordinate: Camera Coordinate: Image Plan f p1p1 p2p2 p3p3 p4p4 C cycy cxcx czcz { C } v u { I } (u o, v o ) is image principal point Optical Axis

11. Areas of triangles (Given): Volumes of tetrahedra: Use unit vector c u i to represent c P i Pose Estimation (Step 2) C cycy cxcx czcz { C } P1P1 P3P3 P2P2 P4P4 h cP1cP1 cP2cP2 cP4cP4 cP3cP3 cuicui (From Step 1)

12. Pose Estimation (Step 2) Use vectors to calculate Volume: Express d 2, d 3, d 4 as a function of d 1 : C cycy cxcx czcz { C } P1P1 P3P3 P2P2 P4P4 h cP1cP1 cP2cP2 cP4cP4 cP3cP3 cuicui

13. Pose Estimation (Step 2) Use a line segment s 1k to compute squared distance: Use parametric representation and simplify C cycy cxcx czcz { C } P1P1 P2P2 cP1cP1 cP2cP2 cu1cu1 cu2cu2 s 12

14. Pose Estimation (Step 2) Substitute d 1 into the following equation to obtain the 3D coordinates of the feature points:

15. Transformation Matrix w T o is given Transformation matrix o T c can be obtained by step 2 Pose Estimation (Step 3) cycy cxcx czcz { C } wywy wxwx wzwz { W } oyoy oxox ozoz { O } wTowTo oTcoTc { W } : World Coordinate { O } : Object Coordinate { C } : Camera Coordinate

16. Pose Estimation (Step 4) The Final Pose of camera can be solved cycy cxcx czcz { C } wywy wxwx wzwz { W } oyoy oxox ozoz { O } wTowTo oTcoTc wTcwTc

17. Current Results of VTS Experimental Setup Motion Recording Computer Webcam Feature Ruler

18. Current Results of VTS

19. Conclusion Heavily depend on image points -Increase image resolution (Now using 640 X 480 pixels) Use some optimization methods to increase accuracy -Gauss-Newton Line search method

20. Future Plan Develop this method and test the performance Try to fuse the data with the µIMU data Develop the optimization method after finishing data fusion

21. Reference [1]Abidi M.A., Chandra T., “A new efficient and direct solution for estimation using quadrangular targets: algorithm and evaluation,” IEEE transactions on pattern analysis and machine intelligence, Vol.17, No.5, pp.534-538, 1995. [2] Abidi M.A., Chandra T., “Pose estimation for camera calibration and landmark tracking,” IEEE International Conference on Robotics and Automation, 2009. [3]Forsyth Ponce, “Computer Vision: A Modern Approach,” Prentice Hall, 2003

22. Thanks for your attention