Kiera Henning, Jiwon Kim, Lost and Found The Distance Transform Kiera Henning, Jiwon Kim,

Pictorial Structures Represent the ‘lost’ item with a tree G = (V, E). V = {parts of the object} E = {deformations between parts} [Felzenszwalb, Huttenlocher 00]

An example template tree

The Problem Given an object, find the best location of the object in the image while respecting the structure of the object defined by the tree.

The Problem (cont) Define For a given part, j, if we know the location of our parent, i, then the best location for j is that which minimizes

The Solution Beginning with the leaves and continuing toward the root one depth at a time, determine B. If all nodes at depth at least d have already determined B, then all nodes at depth d-1 can determine B.

Deformation cost Suppose

Transformed Grid Let location , and define , where

Transformed Grid (cont) Then Run DT on grid defined by T.

Match cost Using PCA, can initialize m to measure the distance from space spanned by eigenvectors. Using simple color scheme, m measures how well the area matches the average color and variance of the part. Using sum of squared differences to match a part expected to change very little.

Examples Using PCA Success! Failed…

Examples Using PCA

Example Using Sum Sq Diff

Document Recognition Documents on Desk Document Image

Document Template SIFT features [Lowe 04]

Example

Feature Points

Distance Transform

chamf(A,B) = ∑a∈A minb∈B ||a-b|| Approach 1 L22 distance defined in 2D feature space [px,py] (2D location x, y) Assymetric Chamfer distance chamf(A,B) = ∑a∈A minb∈B ||a-b|| Brute-force search for best matching translation and rotation (x,y,θ) between A and B

Result Sparse document incorrectly matched to more dense document

Approach 2 L22 distance defined in 2D feature space [px,py] (2D location x, y) Symmetric Chamfer distance chamf(A,B) + chamf(B,A) Brute-force search for best matching translation and rotation (x,y,θ) between A and B

Result H = 268.7 H = 239.3 Non-existing document scores only slightly lower than existing document

Approach 3 L22 distance defined in 3D feature space [px,py,αpo] (x, y + edge orientation) Symmetric Chamfer distance chamf(A,B) + chamf(B,A) Brute-force search for best matching translation and rotation (x,y,θ) between A and B  Better at answering existence query

Result H = 673.1 H = 267.0 Non-existing document scores clearly lower than existing document

Approach 4 L2 distance defined in 3D feature space [px,py,αpo] (x, y + edge orientation) Symmetric, partial Hausdorff distance h(A,B) = K-tha∈A minb∈B ||a-b|| H(A,B) = max(h(A,B), h(B,A)) Efficient search for best matching translation, rotation and scale (x,y,θ) between A and B under partial occlusion

Partial Hausdorff Distance h(A,B) = K-tha∈A minb∈B ||a-b|| Rank a’s according to distance to closest b K=|A| equivalent to h(A,B) = max… 1-K/|A| dictates amount of partial occlusion allowed

Fast Hausdorff Search (1) H(A,B) decreases linearly  can compute lower bound for a cell Subdivide and prune in transformation space [tx,ty,sx,sy] [Huttenlocher et al. 92]

Fast Hausdorff Search (2) Additional acceleration tricks Early rejection N = # of points in A whose dtrans value > thresh If |A| > N > K, skip rest of points for the given cell Early acceptance Probe a randomly chosen fraction of A (e.g., 20%) If fraction of points < thresh larger than K/|A|, accept cell Skipping forward [Huttenlocher et al. 92]

Result Speedup Test image: Template: Xform space: 1600x1200 ~10,000 features Template: ~600x500 ~1,000 features Xform space: 36 rotation bins 100 scales between 0.5 and 1.0 (tx,ty,θ) (tx,ty,θ,sx,sy) Exhaustive search > 10 minutes ? Fast Hausdorff search < 1 minute 4-5 minutes

Result Partial occlusion (success, failure) works up to ~20% occlusion

Discussion Comparison w/ SIFT-based recognition Accuracy (error rate, occlusion handling) Inherently lower, because full feature descriptor information not used Efficiency (speed) Fast multi-resolution search performs quite well Potentially could be made faster by using fewer features However, SIFT-based method also has room for speedup

References Daniel Huttenlocher, William Rucklidge. A Multi-Resolution Technique for Comparing Images Using the Hausdorff Distance. Technical Report TR 92-1321, 1992. Pedro Felzenszwalb, Daniel Huttenlocher. Efficient Matching of Pictorial Structures. In CVPR, 2000. Martin Handford. Where’s Waldo Now?