1Ellen L. Walker Matching Find a smaller image in a larger image Applications Find object / pattern of interest in a larger picture Identify moving objects between a pair of pictures Pre-process for multi-image recognition, e.g. ster You can’t just subtract! Noise Occlusion distortion

2Ellen L. Walker A simple matching algorithm For each position / location / orientation of the pattern Apply matching criteria Local maxima of the criteria exceeding a preset threshold are considered matches (Sound familiar? Replace “match criteria” with “edge strength…”)

3Ellen L. Walker Match Criteria: Digital Difference Take absolute value of pixel differences across the mask Convolution with mask of all -1’s followed by absolute value Not very robust to noise & slight variations Best is 0, so algorithms would minimize rather than maximize Very dark or very light masks will give larger mismatch values!

4Ellen L. Walker Match Criteria (Correlations) All are inverted (and 1 is added to avoid divide by 0) C1 - use largest distance in mask C2 - use sum of distances in mask C3 - use sum of squared distances in mask

5Ellen L. Walker Matching Strategies Brute force (try all possibilities) Hierarchical (coarse-to-fine) Hierarchical (sub-patterns; graph match) Optimized operation sequence Short-circuit processing at mismatches for efficiency

6Ellen L. Walker Matching Higher-level features Elementary geometric properties (boundaries) Boundary length (perimeter) Curvature: perimeter/ count of “direction change” pixels Chord distribution (lengths & angles of all chords) Complete boundary representations Polygonal approximation (recursive line-split algorithm) Contour partitioning (curvature primal sketch) B-Splines

7Ellen L. Walker When do they match? Exact (numerical) match Match within (numerical) tolerance, i.e. threshold numeric difference Match within (geometric) tolerance, i.e. threshold geometric distance to boundary More complex descriptions are harder to match; generally more sensitive to noise!

8Ellen L. Walker Geometric Invariants Given: known shape and known transformation Use: measure that is invariant over the transformation The value is measurable and constant over all xformed shapes Examples Euclidean distance: invariant under translation & rotation Angle between line segments: translation, rotation, scale Cross-ratio: projective invariants (including perspective)

9Ellen L. Walker Geometric Transformations In general, a geometric transformation is any operation on points that yields points Linear transformations can be represented by matrix multiplication: Result is x’/s’, y’/s’

10Ellen L. Walker Example transformations Translation Set diagonals to 1, right column to new location, all else 0 Rotation Set upper four elements to cos(theta), -sin(theta), sin(theta), cos(theta), lower diagonal to 1, all else 0 Scale Set diagonals to 1 and lower right to scale factor Projective transform Any arbitrary 3x3 matrix!

11Ellen L. Walker Cross Ratio Consider four rays “cut” by two lines I = (A-C)(B-D) / (A-D)(B-C)

12Ellen L. Walker Cross Ratio Examples Two images of one object makes 2 matching cross ratios! Dual of cross ratio: four lines from a point instead of four points on a line Any five non-collinear but coplanar points yield two cross-ratios (from sets of 4 lines)

13Ellen L. Walker Using Invariants for Recognition Measure the invariant in one image (or on the object) Find all possible instances of the invariant (e.g. all sets of 4 collinear points) in the (other) image If any instance of the invariant matches the measured one, then you (might) have found the object Research question: to what extent are invariants useful in noisy images?