Download presentation
Presentation is loading. Please wait.
Published byLoren Terry Modified over 8 years ago
1
Digital Camera and Computer Vision Laboratory Department of Computer Science and Information Engineering National Taiwan University, Taipei, Taiwan, R.O.C. Computer and Robot Vision II Chapter 18 Object Models And Matching Presented by: 傅楸善 & 徐子凡 0989306249 r98922132@ntu.edu.tw 指導教授 : 傅楸善 博士
2
DC & CV Lab. CSIE NTU 18.1 Introduction object recognition: one of most important aspects of computer vision
3
DC & CV Lab. CSIE NTU Joke
4
DC & CV Lab. CSIE NTU 18.2 Two-Dimensional Object Representation 2D shape analysis useful in machine vision application: medical image analysis aerial image analysis manufacturing
5
DC & CV Lab. CSIE NTU 18.2 Two-Dimensional Object Representation 2D shape representation classes: 18.2.1 global features 18.2.2 local features 18.2.3 boundary description 18.2.4 skeleton 18.2.5 2D parts
6
DC & CV Lab. CSIE NTU 18.2.1 Global Feature Representation 2D object: can be thought of as binary image value 1: pixels of object value 0: pixels outside object 2D shape features: area, perimeter, moments, circularity, elongation
7
DC & CV Lab. CSIE NTU 18.2.1 Global Feature Representation Shape Recognition by Moments : binary image function : 2D shape digital th moment of : area of S number of pixels of S
8
DC & CV Lab. CSIE NTU 18.2.1 Global Feature Representation moment invariants are functions of digital moments invariant under certain shape transformations. translation, rotation, scaling, skew center of gravity of S:
9
DC & CV Lab. CSIE NTU 18.2.1 Global Feature Representation central th moment of S: central moments: translation invariant normalized central moments of S:
10
DC & CV Lab. CSIE NTU 18.2.1 Global Feature Representation seven functions that are rotation invariant
11
DC & CV Lab. CSIE NTU Original Half Size MirroredRotated 2°Rotated 45°
12
DC & CV Lab. CSIE NTU 18.2.1 Global Feature Representation Fourier descriptors: another way for extracting features from 2D shapes defined to characterize boundary The main idea is to represent the boundary as a function of one variable, expand in its Fourier series, and use the coefficients of the series as Fourier descriptors (FDs). finite number of FDs: can be used to describe the shape
13
DC & CV Lab. CSIE NTU 18.2.1 Global Feature Representation Each coordinate pair can be treated as a complex number so that Discrete Fourier transform of is
14
DC & CV Lab. CSIE NTU 18.2.1 Global Feature Representation The complex coefficients are called the Fourier descriptors of the boundary. The inverse Fourier transform of these coefficients restores. Suppose, only the first P coefficients are used.
15
DC & CV Lab. CSIE NTU 18.2.1 Global Feature Representation
16
DC & CV Lab. CSIE NTU 18.2.1 Global Feature Representation Some basic properties of Fourier descriptors. Notation: Impulse function :
17
DC & CV Lab. CSIE NTU Joke
18
DC & CV Lab. CSIE NTU 18.2.2 Local Feature Representation 2D object characterized by: local features, attributes, relationships most commonly used local features: Holes found by connected component procedure followed by boundary tracing detected by binary mathematical morphology, if hole shapes known properties: areas, shapes Corner detection: can be performed on binary or gray tone image property: angle at which lines meet
19
DC & CV Lab. CSIE NTU Joke
20
DC & CV Lab. CSIE NTU 18.2.3 Boundary Representation boundary representation: most common representation for 2D objects. 3 main ways to represent object boundary: sequence of points chain code sequence of line segments
21
DC & CV Lab. CSIE NTU 18.2.3 Boundary Representation The Boundary as a Sequence of Points boundary points from border-following or edge- tracking algorithms interest points: boundary points with special property useful in matching
22
DC & CV Lab. CSIE NTU 18.2.3 Boundary Representation The Chain Code Representation chain encoding: can be used at any level of quantization saves space required for row and column coordinates boundary encoded: first quantized by placing over square grid square grid side length: determines resolution of encoding marked points: grid intersections closest to curve and used in encoding * : marks starting point of curve
23
DC & CV Lab. CSIE NTU chain encoding of boundary curve
24
DC & CV Lab. CSIE NTU 18.2.3 Boundary Representation line segments: links: to be used to approximate the curve encoding scheme: eight possible directions assigned integer between 0, 7 chain: chain encoding: in the form
25
DC & CV Lab. CSIE NTU 18.2.3 Boundary Representation length of chain code with n chains: can be simply estimated as n n o : number of odd chain codes n e : number of even chain codes n c : number of corners L: unbiased estimate of perimeter length Freeman suggested:
26
DC & CV Lab. CSIE NTU 18.2.3 Boundary Representation The Boundary as a Sequence of Line Segments line segment sequence: after boundary segmented into near-linear portion line segment sequence: used in shape recognition or other matching tasks : coordinate location where pair of lines meet : angle magnitude where pair of lines meet sequence of junction points to represent line segment sequence
27
DC & CV Lab. CSIE NTU 18.2.3 Boundary Representation sequence of junction points representing test object T an association goal: given O, T, to find F satisfying i < j F(i) < F(j) or F(i) = missing or F(j) = missing
28
DC & CV Lab. CSIE NTU Joke
29
DC & CV Lab. CSIE NTU 18.2.4 Skeleton Representation strokes: long, sometimes thin parts forming shapes
30
DC & CV Lab. CSIE NTU 18.2.4 Skeleton Representation symmetric axis transform: set of maximal circular disks that fit inside object symmetric axis: locus of centers of these maximal disks
31
DC & CV Lab. CSIE NTU 18.2.4 Skeleton Representation The symmetric axis is one example of a skeleton description of 2D object. symmetric axis is not always completely representative of the strokes of an object. rectangle: consists of five line segments not single line symmetric axis: extremely sensitive to noise make it difficult to use in matching.
32
DC & CV Lab. CSIE NTU 18.2.4 Skeleton Representation local symmetry: midpoint P of line segment BA α : angle between BA and outward normal N a at A α : angle between BA and inward normal N b at B
33
DC & CV Lab. CSIE NTU 18.2.4 Skeleton Representation The loci of local symmetries that are maximal w.r.t. forming a smooth curve are called axes or spines. cover of axis: portion of shape subtended by axis axis cover properly contained in another cover: second axis subsumes first The short diagonal axes are subsumed by the horizontal and vertical axes and can be either deleted or relegated to a lower place in a hierarchical description of the shape (Chap. 19).
34
DC & CV Lab. CSIE NTU 18.2.4 Skeleton Representation Axes of smoothed local symmetries of several objects.
35
DC & CV Lab. CSIE NTU Joke
36
DC & CV Lab. CSIE NTU 18.2.5 Two-Dimensional Part Representation parts, attributes, interrelationships: form structural description of shape nuclei: regions where primary convex subset overlap nuclei
37
DC & CV Lab. CSIE NTU 18.2.5 Two-Dimensional Part Representation near-convexity: allows noisy distorted instances to have same decompositions P 1, P 2 : two points on object boundary L I relation: visibility relation if line completely interior to object boundary, Apply the graph-theoretic clustering algorithm to determine clusters of visibility relation
38
DC & CV Lab. CSIE NTU 18.2.5 Two-Dimensional Part Representation decomposition of three similar shapes into near- convex pieces
39
DC & CV Lab. CSIE NTU Joke
40
DC & CV Lab. CSIE NTU 18.3 Three-Dimensional Object Representations 18.3.1 Local Features Representation. 18.3.2 Wire Frame Representation. 18.3.3 Surface-Edge-Vertex Representation. 18.3.4 Stick, Plates, and Blobs. 18.3.5 Generalized Cylinder Representation. 18.3.6 Super-quadric Representation. 18.3.7 Octree Representation. 18.3.8 The Extended Gaussian Image. 18.3.9 View-Class Representation.
41
DC & CV Lab. CSIE NTU 18.3.1 Local Features Representation Local Features Representation range data: obtained from laser range finder, light striping, stereo, etc. from depth, try to infer surfaces, edges, corners, holes, other features 3D matching more difficult than 2D because of occlusion
42
DC & CV Lab. CSIE NTU Joke
43
DC & CV Lab. CSIE NTU 18.3.2 Wire Frame Representation wire frame model: 3D object model with only edges of object
44
DC & CV Lab. CSIE NTU 18.3.2 Wire Frame Representation two-color hyperboloid and its line drawing
45
DC & CV Lab. CSIE NTU 18.3.2 Wire Frame Representation Necker cube: lower-vertical face or upper-vertical face closer to viewer Schroder staircase: viewed either from above or from below
46
DC & CV Lab. CSIE NTU 18.3.2 Wire Frame Representation
47
DC & CV Lab. CSIE NTU 18.3.2 Wire Frame Representation
48
DC & CV Lab. CSIE NTU 18.3.2 Wire Frame Representation general-viewpoint assumption: none of the following situations 1. two vertices of scene objects represented at same picture point 2. two scene edges seen as single line in picture 3. vertex seen exactly in line with unrelated edge
49
DC & CV Lab. CSIE NTU 18.3.2 Wire Frame Representation general-viewpoint assumption: heart of line-drawing interpretation viewpoint in perspective projection: center of projection viewpoint in orthographic projection: direction of projection
50
DC & CV Lab. CSIE NTU subjective contours of Kanizsa: white occluding triangle in space 18.3.2 Wire Frame Representation
51
DC & CV Lab. CSIE NTU
52
DC & CV Lab. CSIE NTU 18.3.2 Wire Frame Representation line labels for visible projections of surface-normal discontinuities:
53
DC & CV Lab. CSIE NTU Joke
54
DC & CV Lab. CSIE NTU 18.3.3 Surface-Edge-Vertex Representation VISIONS system: Visual Integration by Semantic Interpretation of Natural Scenes PREMIO system: Prediction in Matching Images to Objects PREMIO 3D object model: hierarchical, relational model with five levels world, object, face/edge/vertex, surface/boundary, arc/2D, 1D piece
55
DC & CV Lab. CSIE NTU 18.3.3 Surface-Edge-Vertex Representation world level: arrangement of different objects in world object level: arrangement of different faces, edges, vertices forming objects face level: describes face in terms of surfaces and boundaries surface level: specifies elemental pieces forming surfaces
56
DC & CV Lab. CSIE NTU 18.3.3 Surface-Edge-Vertex Representation 2D piece level: describes pieces and specifies arcs forming boundaries 1D piece level: describes elemental pieces forming arcs SDS: spatial data structure A/V: attribute-value table
57
DC & CV Lab. CSIE NTU
58
DC & CV Lab. CSIE NTU Joke
59
DC & CV Lab. CSIE NTU 18.3.4 Sticks, Plates, and Blobs sticks, plates, blobs model: rough models of 3D objects used in rough-matching near-convex sticks: long, thin parts with only one significant dimension cannot bend very much two logical endpoints set of interior points center of mass
60
DC & CV Lab. CSIE NTU 18.3.4 Sticks, Plates, and Blobs plates: flattish wide parts with two nearly flat surfaces two significant dimensions cannot fold very much set of edge points, set of surface points, center of mass blobs: parts with three significant dimensions can be bumpy but cannot have concavities set of surface points and center of mass
61
DC & CV Lab. CSIE NTU
62
DC & CV Lab. CSIE NTU 18.3.4 Sticks, Plates, and Blobs attribute-value table: contains global attributes simple-parts relation: lists the parts and their attributes connects-supports relation: gives connections between pairs of parts triples relation: specifies connections between three parts at a time parallel relation: lists pairs of parts that are parallel perpendicular relation: lists pairs of parts that are perpendicular TYPE: 1 for stick, 2 for plate, 3 for blob
63
DC & CV Lab. CSIE NTU
64
DC & CV Lab. CSIE NTU Joke
65
DC & CV Lab. CSIE NTU 18.3.5 Generalized Cylinder Representation generalized cylinder: volumetric primitive defined by axis and cross-section cross section: swept along axis, creating a solid e.g. actual cylinder: generalized cylinder whose axis is straight- line segment and whose cross section is circle of constant radius e.g. cone: generalized cylinder whose axis is straight-line segment and cross section is circle with radius initially zero to maximum
66
DC & CV Lab. CSIE NTU 18.3.5 Generalized Cylinder Representation e.g. rectangular solid: generalized cylinder whose axis is straight line segment and cross section is constant rectangle e.g. torus: generalized cylinder whose axis is circle and whose cross section is constant circle generalized cylinder representation: uses generalized cylinders as primitives torus
67
DC & CV Lab. CSIE NTU 18.3.5 Generalized Cylinder Representation surface-edge-vertex model: very precise sticks-plates-and-blobs model: very rough generalized cylinder model: somewhere in between
68
DC & CV Lab. CSIE NTU 18.3.5 Generalized Cylinder Representation person: modeled roughly as cylinders for head, torso, arms, legs dotted lines: axes of cylinders
69
DC & CV Lab. CSIE NTU Joke
70
DC & CV Lab. CSIE NTU 18.3.6 Super-quadric Representation Super-quadrics: lumps of clay deformable and can be glued into object models Super-quadric models: mainly used with range data
71
DC & CV Lab. CSIE NTU 18.3.6 Super-quadric Representation Super-quadrics are a flexible family of 3-dimensional parametric objects, useful for geometric modeling. By adjusting a relatively few number of parameters, a large variety of shapes may be obtained.
72
DC & CV Lab. CSIE NTU Figure 18.13 Range data image of (a) a doll, (b) its super-quadric fit (c), (d) wire frame
73
DC & CV Lab. CSIE NTU Joke
74
DC & CV Lab. CSIE NTU 18.3.7 Octree Representation octree encoding: geometric modeling technique used to represent 3D objects used in computer vision, robotics, computer graphics octree hierarchical: 8-ary tree structure each node in octree corresponds to cubic region of universe
75
18.3.7 Octree Representation DC & CV Lab. CSIE NTU
76
DC & CV Lab. CSIE NTU 18.3.7 Octree Representation full, empty, partial full: if cube is completely enclosed by 3D object empty: if cube contains no part of object partial: if cube partly intersects object partial: has eight children representing partition of cube into octants labeled full or empty : no children
77
DC & CV Lab. CSIE NTU
78
DC & CV Lab. CSIE NTU Joke
79
DC & CV Lab. CSIE NTU 18.3.8 The Extended Gaussian Image 3D object: collection of surface normals, one at each point of object surface planar surface: all points on surface map to same surface normal convex with positive curvature everywhere: distinct surface normal everywhere set of surface normals can be mapped to a unit sphere (Gaussian sphere) by placing tail at center head outward Gaussian image of object: resultant set of points on Gaussian sphere
80
DC & CV Lab. CSIE NTU
81
DC & CV Lab. CSIE NTU 18.3.8 The Extended Gaussian Image for planar objects: Gaussian image not invertible, not precise enough for use δO: small surface patch of object δS: corresponding surface patch on Gaussian sphere Gaussian curvature K:
82
DC & CV Lab. CSIE NTU 18.3.8 The Extended Gaussian Image (ξ,η): point on Gaussian sphere corresponding to point (u, v) on object surface extended Gaussian image: planar region: Gaussian curvature 0, point mass in extended Gaussian image
83
DC & CV Lab. CSIE NTU Joke
84
DC & CV Lab. CSIE NTU 18.3.9 View-Class Representation view classes: each representing set of viewpoints sharing some property same object surfaces visible same line segments visible relational distances between relational structures are similar characteristic views: sets producing topologically isomorphic line drawings
85
DC & CV Lab. CSIE NTU 18.3.9 View-Class Representation three view classes of cube producing topologically isomorphic line drawings
86
DC & CV Lab. CSIE NTU
87
DC & CV Lab. CSIE NTU
88
DC & CV Lab. CSIE NTU 18.3.9 View-Class Representation aspect graph of object: graph structure where 1. each node represents topologically distinct view of object 2. a node for each such view of object 3. each arc represents a visual event at transition 4. there is an arc for each such transition
89
DC & CV Lab. CSIE NTU
90
DC & CV Lab. CSIE NTU Joke
91
DC & CV Lab. CSIE NTU 18.4 General Frameworks for Matching matching: finding correspondence between two entities consistent labeling procedures: examples of matching algorithms
92
DC & CV Lab. CSIE NTU 18.4 General Frameworks for Matching 18.4.1 Relational-Distance Approach to Matching 18.4.2 Ordered Structural Matching 18.4.3 Hypothesizing and Testing with Viewpoint Consistency Constraint 18.4.4 View-Class Matching 18.4.5 Affine-Invariant Matching
93
DC & CV Lab. CSIE NTU 18.4.1 Relational-Distance Approach to Matching relational distance: compares two structures and determines similarity Relational-Distance Definition D x : relational description D x = {R 1, …, R I } : sequence of relations X : set of parts of entity being described R i : relation indicating various relationships among parts D A : relational description with part set A D B : relational description with part set B
94
DC & CV Lab. CSIE NTU 18.4.1 Relational-Distance Approach to Matching assumption: |A| = |B|, otherwise add dummy parts to smaller set f: any one-one, onto mapping from A to B N: positive integer composition R 。 F of relation with function f:
95
DC & CV Lab. CSIE NTU 18.4.1 Relational-Distance Approach to Matching f: maps parts from set A to parts from set B structural error of f for Ith pair of corresponding relations in D A, D B : total error of f with respect to D A, D B : relational distance GD( D A, D B ) between D A, D B :
96
DC & CV Lab. CSIE NTU 18.4.1 Relational-Distance Approach to Matching best mapping from D A to D B : mapping f that minimizes total error
97
DC & CV Lab. CSIE NTU best mapping from to is for this mapping:
98
DC & CV Lab. CSIE NTU
99
DC & CV Lab. CSIE NTU
100
DC & CV Lab. CSIE NTU
101
DC & CV Lab. CSIE NTU 18.4.1 Relational-Distance Approach to Matching Relational Distance as a Metric relational distance: used to determine similarity of unknown object to an object model can also be used to compare object models to grouping models in a large database f relational isomorphism: if f one-one, onto from A to B and E(f) = 0 f: A → B relational isomorphism: D A, D B isomorphic GD: relational-distance measure
102
DC & CV Lab. CSIE NTU 18.4.1 Relational-Distance Approach to Matching D A, D B, D C : metric property of GD:
103
DC & CV Lab. CSIE NTU 18.4.1 Relational-Distance Approach to Matching Attributed Relational Descriptions and Relational Distance extend relational description and relational distance to include properties of parts properties of the whole properties of these relationships
104
DC & CV Lab. CSIE NTU Joke
105
DC & CV Lab. CSIE NTU 18.4.2 Ordered Structural Matching definition of ordering on primitives: greatly reduces complexity of search
106
DC & CV Lab. CSIE NTU Joke
107
DC & CV Lab. CSIE NTU 18.4.3 Hypothesizing and Testing with Viewpoint Consistency Constraint viewpoint consistency constraint: The locations of all projected model features in an image must be consistent with projection from a single viewpoint.
108
DC & CV Lab. CSIE NTU Joke
109
DC & CV Lab. CSIE NTU 18.4.4 View-Class Matching if 3D object represented by view-class model, matching divided into 2 stages: 1. determining view class of object 2. determining precise viewpoint within that view class
110
DC & CV Lab. CSIE NTU 18.4.4 View-Class Matching relational pyramid: hierarchical relational structure to represent view class Level-1 primitives: straight- and curved-line segments Level-2 relations: junctions and loops Level-3 relations: adjacency, collinearity, junction parallelness, loop-inside-loop
111
DC & CV Lab. CSIE NTU 18.4.4 View-Class Matching Pose Determination within View Class relational pyramid: hierarchical, relational structure to constrain matching
112
DC & CV Lab. CSIE NTU Joke
113
DC & CV Lab. CSIE NTU 18.4.5 Affine-Invariant Matching set of interest points lying in z = z 0 plane rotation matrix relating model reference frame to camera reference frame: translation of object reference frame to camera reference frame:
114
DC & CV Lab. CSIE NTU 18.4.5 Affine-Invariant Matching f: distance between image plane and center of perspectivity : observed image data points by perspective projection: when translation t 3 in z -direction large compared with r 31 x m + r 32 y m :
115
DC & CV Lab. CSIE NTU 18.4.5 Affine-Invariant Matching A : 2 x 2 (scaling, rotation, skewing) matrix b : 2D (translation) vector affine 2D correspondence: Aw + b
116
DC & CV Lab. CSIE NTU 18.4.5 Affine-Invariant Matching necessary and sufficient to define plane uniquely: 3 noncollinear points
117
DC & CV Lab. CSIE NTU 18.4.5 Affine-Invariant Matching The Hummel-Wolfson-Lamdan Matching Algorithm to match noncollinear triplets in model interest points with scene: Step 1: preprocessing: convert model interest points into affine- invariant model Step 2: recognition: match model against image using affine representation
118
DC & CV Lab. CSIE NTU 18.4.5 Affine-Invariant Matching Shortcomings of the Affine-Invariant Matching Technique affine-invariant matching technique: mathematically sound in noiseless case shortcomings of affine-invariant matching in practice: 1. if three noncollinear points not numerically stable, points not reliable 2. coordinates of detected interest points: noisy in real image 3. partial object symmetries may cause wrong matching
119
DC & CV Lab. CSIE NTU Joke
120
DC & CV Lab. CSIE NTU 18.5 Model Database Organization organize database of models: to allow rapid access to most likely candidate group similar relational models into clusters and choose representative arrows: indicate mapping from parts of object 2 to parts of other objects
121
DC & CV Lab. CSIE NTU
122
DC & CV Lab. CSIE NTU END
Similar presentations
© 2024 SlidePlayer.com Inc.
All rights reserved.