Computer Vision Laboratory 1 Unrestricted Recognition of 3-D Objects Using Multi-Level Triplet Invariants Gösta Granlund and Anders Moe Computer Vision Laboratory Linköping University SWEDEN

Computer Vision Laboratory 2 By unrestricted, we imply that the recognition shall be done independently of object position, scale, orientation and pose, against a structured background. It shall not assume any preceding segmentation and allow a reasonable degree of occlusion.

Computer Vision Laboratory 3 Traditional approach

Computer Vision Laboratory 4 Object Representation

Computer Vision Laboratory 5 Object Parameters VarObject characterization ωObject class xHorisontal position of object yVertical position of object Horisontal pose angle of object θVertical pose angle of object ψOrientation of object in image plane sScale or size of object

Computer Vision Laboratory 6 Object Identification -- An Inverse Problem Implies a two-step process: 1.Postulation of a certain model 2.Performing measurements, and comparing these with a reference, under the assumption of the particular model

Computer Vision Laboratory 7 Requirements of Model Structure Models shall be fragmentable such that a certain model can be part of a more complex or higher order model. Due to this recursive character, we will simply denote them all models, be it parts or combinations. Learning of models shall proceed from lower levels to higher levels. Acquired lower level models shall be usable as parts of several different higher order models. A particular model is only acquired once, and its first occurrence is used as the representation of that model.

Computer Vision Laboratory 8 Invariance to illumination

Computer Vision Laboratory 9 Local versus global properties

Computer Vision Laboratory 10 Conflicting interpretations

Computer Vision Laboratory 11 Object Representation

Computer Vision Laboratory 12 Compact description of regions

Computer Vision Laboratory 13 Edges and lines

Computer Vision Laboratory 14 Curvature, corners I

Computer Vision Laboratory 15 Curvature, corners II

Computer Vision Laboratory 16 Triplet Models

Computer Vision Laboratory 17 Multi-level Triplet Models

Computer Vision Laboratory 18 Properties of Triplet Structure It allows a unique ordering of the feature points, which is implemented such that the triplet is ”right oriented”, i.e. that the angle α < π. The triplet structure allows us to define a scale invariant structure parameter The distance between the two feature points not connected by the triplet, must be shorter than the two other distances between feature points. The triplet can be brought into a ”normal orientation” by aligning leg to make

Computer Vision Laboratory 19 Invariance Properties The preceding properties together with the hierarchical arrangement of triplets make the following parameter variations trivial: Orientation in the image plane Scale Object position in x and y

Computer Vision Laboratory 20 Procedures Statistics Assumption of Structure Preselection Grouping

Computer Vision Laboratory 21 Examples of Grouping Rules Spatial grouping range: We expect primitives to increase in spatial size going towards higher levels. Object closure criteria: Tests for homogeneity such as similar density or color inside the triplets, to indicate parts of a common object or region. Symmetry

Computer Vision Laboratory 22 Channel Inform. Representation

Computer Vision Laboratory 23 Triplet Components A triplet can be characterized in a number of equivalent fashions. The components used are: : Point feature vectors, k=1,2,3, each one coded with h f channels. : Angle between triplet legs 1 and 2, coded with h a channels. : Relative length of triplet legs, coded with h g channels.

Computer Vision Laboratory 24 Triplet Vector

Computer Vision Laboratory 25 Mapping Onto Object States

Computer Vision Laboratory 26 Dynamic Binding Variables From the purely geometric and feature related entities, it is desired to map into variables that are object-related: They change as a consequence of manipulation of the object, which is essential They can be expected to be shared with, or at least coupled to, other primitives at the same level, or at a different level

Computer Vision Laboratory 27 Object States

Computer Vision Laboratory 28 Mapping from Features to Objects

Computer Vision Laboratory 29 Features for Higher Level Triplets

Computer Vision Laboratory 30 Mapping from Features to Objects

Computer Vision Laboratory 31 Consistency Check of Outputs

Computer Vision Laboratory 32 Removal of Multiple Models

Computer Vision Laboratory 33 Estimation of Object Orientation Orientation estimates of an object are obtained as the difference between the observed orientation of the higher level triplets, and their estimates of the original orientation at training.

Computer Vision Laboratory 34 Estimation of Scale Derived estimate of scale is divided by the length of the original triplet.

Computer Vision Laboratory 35 Object Training Setup

Computer Vision Laboratory 36 Object scanner

Computer Vision Laboratory 37 Training Sequence

Computer Vision Laboratory 38 Estim. Pose, Orientation and Scale

Computer Vision Laboratory 39 Average Error for 60° Rotated and 10% Rescaled Object in Test Set EstimateAverage Error Pose-x1.8° Pose-y1.8° Orientation6.3° Scale4.9%

Computer Vision Laboratory 40 Scaled and Occluded Object

Computer Vision Laboratory 41 Multiple, Occluded Objects

Computer Vision Laboratory 42 Estimates of Pose