1. International Conference on Computer Vision and Graphics, ICCVG ‘2002 Algorithm for Fusion of 3D Scene by Subgraph Isomorphism with Procrustes Analysis Krzysztof Skabek, Przemysław Kowalski Instytut Informatyki Teoretycznej i Stosowanej PAN, ul. Bałtycka 5, 44-100 Gliwice e-mail: krzysiek@iitis.gliwice.pl

2. Contents 1. Active vision 2. Stages of 3D fusion 3. Graph representation and algorithms 5. Matching structure graphs 4. Algorithm for 3D Fusion 5. Experiments

3. Active Vision Platform Platform movements The observed 3D scene

4. Purpose: Obtaining a complete 3D representation of the scene and relations between components of the scene basing on a set of 3D frames from multiple viewpoints.Assumptions: No a-priori information about objects of the scene. Unknown location of the viewpoint. We focus on polyhedral objects.

5. Architecture of Active Vision Act Sensing Planning camera Mision planning Navigator Pilot Controller Engines Image preprocessing Model integration Location: x,y,z,  3D model Methods of 3D fusion

6. 3D Fusion of Multipoint Views 3D Fusion 3D Fusion – integration process of objects in 3D scene on the basis of visual information from several viewpoints.

7. Stages of 3D Fusion Current view Viewpoint loc. 3D model Corection Vision device Prediction Navigation to a new viewpoint Knowledge of the scene Comparing the view and the model Exact viewpoint loc. Updating the model Hypothesis about the scene objects Checking the completeness

8. Preprocessing of Visual Information  Improvement of image quality, noise reduction  Image segmentation, extraction of lines, segments,vertices: Susan, Hough Methods  Stereo matching, depth map: active contours, hardware support (ranging lasers, depth sensors) Algorithms prepared for Khoros platform

9. Viewpoint parameters  T – vector of translation (3×1),  R – rotation matrix (3×3, orthogonal)  s – scale (scalar) Relation between coordinates of point P: P w – global coordinates, P k – coordinate system of the camera P k = R(P w –T)s

10. Graph representation of 3D scene Contourgraph Face graph 1 2 3 4 1 2 3 4 5 6 7

11. Graph Isomorphism  2 3 3 2 2 3 3 2 Subgraph Isomorphism   2 3 3 2 2 3 3 3 3 2 Weak Subgraph Isomorphism   2 3 3 2 2 3 3 3 3 2 4 4

12. Detection of Graph Isomorphisms  Permutation method  Clique detection method  Ullman method  A* method (error correction)  Decision tree method Algorthm with analysis of 3D structure deformation (decision tree, consistency checking, branch pruning, geometric similarity)

13. Similarity of Shape – Procrustes analysis --- - object A --- - object B --- - A to B matching D 2 (A, B) = || B – S·R·A – T || 2 Translation (T) Scale (S) Rotation (R)

14. Implementation of 3D Fusion (matching contour graphs) Stage I: Generation of groups of vertices (quadruplets) fulfilling conditions: u Procrustes distance <  u Preserving edge topology Stage II: (for eqch group of vertices from stage I) Calculation of local transformation (T L R L S L ) Matching the remaining vertices: u Local Procrustes distance <  u Preserving edge topology Calculation of exact transformation (T R S) 0 2 3 4 1 0 2 3 4 1 5 6 n ISOMORPHISMS V

15. Implementation of 3D Fusion (model updating) A B C D E 5 1 4 3 6 2 F GM i-1 GI i A B C D E F 5 GM i

16. Implementation of 3D Fusion (hypothesis of the scene objects) Introducing unconfirmed elements. Hypothesis of scene objects:  Connecting edges  Closing faces  Connecting partial faces  Ground plane detection  Completing vertical faces

17. Conditions of experiments Total transformation error consists of: rotation, translation, scale Tolerance of rotation (R  – matrix of rotation error): R = R   R  Estimation of rotation error:  = ½ [1 – cos(  ) ] ½ Assumed value of rotation error:  = 0.1 for   16°

18. Experiments

19. Thank you for your attention

20. Graph representation of 3D scene II Contour graph: Face graph: - a set of vertices in the scene - a set of edges between vertices - coordinates (x,y,z) of vertices - a set of faces in the scene - a set of connections between faces - parameters of faces - parameters of connections between faces

21. Implementation of 3D Fusion Input data: GI i – contour graphs for views (i=1..n) T i R i S i – estimated transformation (from navigation unit)  i – transformation tolerance (for navigation unit)  i – observation tolerance (for optical unit) Output data: GM n – Contour graphs of model T i R i S i – computed transformation First step of fusion: GM 1  GI 1

22. Experiment I