1. Vehicle Recognition System
Gerald Dalley
Signal Analysis and Machine Perception Laboratory
The Ohio State University
09 May 2002

2. Recognition Engine Steps
Acquire range images
Determine regions of interest (see Kanu)
Local surface estimation
Surface reconstruction
Affinity measure
Spectral Clustering: Normalized cuts
Graph matching

3. Range Image Acquisition
Model building
Testing

4. Local Surface Estimation
What: (1) Estimate the local surface characteristics (2) at given locations
Why:
(1) Vehicles are made up of large low-order surfaces
(1) Look for groups of points that imply such surfaces
(2) Our sets of range images are BIG
(tank from 10 views has over 220,000 range points)
How…

5. Local Surface Estimation: Point Set Selection
In the region of interest…
Collect range image points into cubic voxel bins
(32x32x32mm right now)
Discard bins that have:
Too few points
Points that do not represent a biquadratic surface well
Retain only the centroids of the bins and their surface fits

6. Local Surface Estimation: Biquadratic Patches
PCA  local coordinate system
Least squares biquadratic fit:
f(u,v) = a1u2 + a2uv + a3v2 + a4u + a5v + a6 w u

7. Surface Reconstruction
Voxel
centroids
Mesh
cocone
See last quarter’s presentation for details on cocone

8. Affinity
Quasi-Definition: Affinity  probability that two mesh points were sampled from the same low-order surface
Why: Can use grouping algorithms to segment the mesh (to make recognition easier)
Our formulation…

9. Affinity, Cont’d. mj qi nj pj qj LaTeX:
\documentclass{slides}\pagestyle{empty}
\newcommand{\ii}{\vspace{-1.25em}\item}
\newcommand{\iii}{\vspace{-0.60em}\item}
\newcommand{\iiii}{\vspace{-0.45em}\item}
\begin{document}
\tiny
\begin{itemize}
\ii \textbf{Basic formulae:}
\iii Symmetric affinity: $A_{ij} = min\left(a_{ij}, a_{ji}\right)$
\iii Asymmetric affinity: $a_{ij} = p_{ij} n_{ij} d_{ij} f_{ij}$
\end{itemize}
\ii \textbf{Where...}
\iii $\sigma_i$ is the RMS$^2$ biquadratic fit error
\iii $\sigma = \sum_{i=1}^{N}{\sigma_i}/N$
\iii $p_{ij} = e^{-\left|\overline{p}_j - \overline{q}_j\right|^2 / \sigma^2}$
\iiii $\overline{p}_j = $ the projection of point
$\overline{q}_j$ onto $i$'s surface estimate
\iii $n_{ij} = e^{-\left(\left(\overline{m}_j.x - \overline{n}_j.x \right)^2 +
\left(\overline{m}_j.y - \overline{n}_j.y \right)^2\right) / 4 \sigma^2}$
\iiii $\overline{m}_j = $ the surface normal
at $\overline{p}_j$, using surface $i$
\iiii $\overline{n}_j = $ the surface normal
at $\overline{q}_j$, using surface $j$
\iii $d_{ij} = e^{-2 \left|\overline{q}_i - \overline{q}_j\right|^2 / r^2}$
\iii $f_{ij} = e^{-\sigma_i^2/ \sigma^2} e^{-\sigma_j^2/ \sigma^2}$
\end{document}

10. Spectral Clustering
Aij is “block diagonal”  Non-zero elements of the 1st eigenvector define a cluster [Weiss,Sarkar96]
y1, where
Ayi=li yi
Aij

11. Spectral Clustering, Cont’d.
Example using our data…
y1, where
Ayi=li yi
Aij
1st Cluster

12. Spectral Clustering: Normalized Cuts
Tend to get disjoint clusters
Need to balance clustering and segmentation

13. = Graph Matching ? For each model i
R: For each unused object segment s
For each (model segment i.t , NULL segment)
Compute penalty for matching s to i.t + all previous matches made
Save this match if it’s better than any other
Recurse to R
Save the best matching of model and object segments for model i
Choose the model with the best match
Partial LaTeX source for match error score (not shown on slide):
\documentclass{slides}\pagestyle{empty}
\begin{document}
\tiny
$N = $ number of segments matched\\
$i = $ segment number\\
$E = 1 - (1-U)^\mu (1-O)^\omega (1-A)^\alpha$\\
$U = \sum_{i=1}^{N} u_i/N, O = \sum_{i=1}^{N} o_i/N, D = \sum_{i=1}^{N} d_i/N $\\
$A = min\left(1, \left|1 - \sum_{i=1}^{N}area(i) / (total\_model\_area) \right|\right) $ \\
$u_i = $ \\
$o_i = \sum_{i=1}^{N-1} | \angle seg\_normal_N, seg\_normal_i - $\\
\hspace*{1.05in} $\angle model\_normal_N, model\_normal_i | \cdot 2 / \pi N $ \\
$d_i = \sum_{i=1}^{N-1} DIST(\left|seg\_centroid_N - seg\_centroid_i\right|, $\\
\hspace*{1.65in} $\left|model\_centroid_N - model\_centroid_i\right|) / N$ \\
\end{document}

14. Graph Matching: Segment Attributes= ?
Unary attributes (for comparing one object segment to one model segment)
Segment area
Mean and Gaussian Curvature
(from a new biquadratic fit to the voxel points participating in the segment)
“Distinctiveness”
Binary attributes (for comparing a pair of object segments to a pair of model segments)
Centroid separation
Angle between normals at the centroid

15. Further Reading
Y. Weiss et al. “Segmentation using eigenvectors: a unifying view”. ICCV: , 1999.
S. Sarkar and K.L. Boyer. “Quantitative measures of change based on feature organization: eigenvalues and eigenvectors”. CVPR 1996.
J. Shi and J. Malik. “Normalized Cuts and Image Segmentation”. PAMI: , 2000.