DEVELOPMENT OF A COMPUTER PLATFORM FOR OBJECT 3D RECONSTRUCTION USING COMPUTER VISION TECHNIQUES Teresa C. S. Azevedo João Manuel R. S. Tavares Mário A. P. Vaz

Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz2 Contents I.Introduction to Computer Vision; II.Computer Platform presentation; III.Experimental results; IV.Conclusions; V.Future work.

Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz3 Computer Vision Introduction Platform Conclusions Future Work Results Computer Vision is continuously trying to develop theories and methods for automatic extraction of useful information from images, as similar as possible to the complex human visual system. Some applications: Medicine - 3D reconstruction / modelling, surgery planning; Identification and navigation systems; Virtual reality; …

Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz4 Goals and Methodology Introduction Platform Conclusions Future Work Results Contactless techniques to recover the 3D geometry of an object are usually divided in two classes: Our goal was to obtain 3D models of objects using an active vision technique called Structure From Motion (SFM). active techniques - require some kind of energy projection or the cameras (or objects) movement to obtain 3D information about the shape; passive techniques - only use ambient light and so, usually, the extraction of 3D information becomes more difficult.

Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz5 Computer Platform Introduction Platform Conclusions Future Work Results Integration of functions for 3D reconstruction, available from five software programs and one computational library, all open source: OpenCV; Peters Matlab Functions; Torrs Matlab Toolkit; KLT; Projective Rectification without Epipolar Geometry; Depth Discontinuities by Pixel-to-Pixel Stereo. Modular structure; Users graphical interface; Computer language: C ++ ; Operational system: Microsoft Windows. Ported to C using MATLAB Compiler toolbox Developing tool: Microsoft Visual Studio, using MFC libraries (Microsoft Foundation Classes);

Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz6 Computer Platform Introduction Platform Conclusions Future Work Results The functions integrated enclose several Computer Vision techniques: feature points detection; feature points matching between two images; epipolar geometry determination; rectification; dense matching. For each technique, the user can easily choose the algorithm to use, as well as conveniently define its parameters.

Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz7 Feature Points detection Introduction Platform Conclusions Future Work Results available algorithms for feature points detection OpenCV KLT Reflect the relevant discrepancies between their intensity values and those of their neighbours; Usually represent vertices of objects, and their detection allows posterior matching between the images of the sequences.

Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz8 Feature Points matching Introduction Platform Conclusions Future Work Results Image 2D points association between sequential images, which are the projection of the same 3D object point; A short set of matching points is enough to determine the epipolar geometry between two images (the fundamental matrix). 1st image feature points coordinates matching points coordinates on 2nd image fundamental matrix available algorithms for feature points matching

Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz9 Feature Points matching Introduction Platform Conclusions Future Work Results Some results:

Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz10 Epipolar Geometry determination Introduction Platform Conclusions Future Work Results Corresponds to the geometrical structure between two stereo images and its expressed mathematically by the fundamental matrix; Also allows the elimination of some previous wrong matches (outliers), as well as make easier the determination of new matching points (dense matching). algorithms for epipolar lines determination algorithms for epipolar geometry determination

Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz11 Epipolar Geometry determination Introduction Platform Conclusions Future Work Results Some results: Epipolar line Inlier

Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz12 Rectification Introduction Platform Conclusions Future Work Results Method that changes two stereo images, in order to make them coplanar; Performing this step makes dense matching easier to obtain; available algorithm for rectification The quality of the results is proportional to the quality of the epipolar geometry determination.

Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz13 Dense matching Introduction Platform Conclusions Future Work Results Disparity map - codifies the distance between the object and the camera(s): closer points will have maximal disparity and farther points will get minimum disparity; A disparity map gives some perception of discontinuity in terms of depth; One of the algorithms also returns a discontinuity map – defines the pixels who border the changing between at least two levels of disparity. available algorithms for dense matching

Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz14 Dense matching Introduction Platform Conclusions Future Work Results Some results: Original images Disparity map Discontinuity map

Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz15 Conclusions Introduction Platform Conclusions Future Work Results The functions, already integrated in the computer platform, give good results when applied to objects with strong characteristics; From the experimental results, it is possible to conclude that low quality results are strongly correlated with few (strong) feature points detection and wrong matching; This weakness is higher as the object shape variation is smooth.

Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz16 Future work Introduction Platform Conclusions Future Work Results The next steps of this work will focus on improving the results obtained when the objects have smooth and continuous surfaces: Finally, the computer platform will be used in 3D reconstruction and characterization of 3D external human shapes. inclusion of space carving techniques for object reconstruction; the feature points to use in the 3D space object definition will be detected with the use of a reduced number of markers added on the object; inclusion of a camera calibration technique, as well as pose and motion estimation algorithms;

DEVELOPMENT OF A COMPUTER PLATFORM FOR OBJECT 3D RECONSTRUCTION USING COMPUTER VISION TECHNIQUES Teresa C. S. Azevedo João Manuel R. S. Tavares Mário A. P. Vaz Acknowledgments This work was partially done in the scope of the project Segmentation, Tracking and Motion Analysis of Deformable (2D/3D) Objects using Physical Principles, reference POSC/EEA- SRI/55386/2004, financially supported by FCT - Fundação para a Ciência e a Tecnologia in Portugal.