1. ISOMAP TRACKING WITH PARTICLE FILTERING
NIKHIL RANE AND STAN BIRCHFIELD
CLEMSON UNIVERSITY
Abstract
Isomap tracking with particle filtering algorithm
Tracking results
The problem of tracking involves challenges like in-plane and out-of-plane rotations, scaling, variations in ambient light and occlusions. In this paper we look at the problem of tracking a person’s head and also estimating its pose in each frame. Robust tracking can be achieved by reducing the dimensionality of high-dimensional training data and using the recovered low-dimensional structure to estimate the state of an object at every time-step with recursive Bayesian filtering. Isometric feature mapping, also known as Isomap, provides an unsupervised framework to find the true degrees of freedom in high-dimensional input data like a person’s head with varying poses. After the data has been reduced to lower dimensions a particle filter can be used to track and at the same time approximate the pose of a person’s head in any image sequence. Isomap tracking with particle filtering is capable of handling rapid translation and out-of-plane rotation of a person’s head with a relatively small amount of training data. The performance of the tracker is demonstrated on an image sequence with a person’s head undergoing translation and out-of-plane rotation.
Algorithm :
Create training set of a person’s face (off-line)
Use Isomap to reduce dimensionality of the training set (off-line)
Run particle filter (Condensation) on test sequence to track the person
Condensation algorithm
Behavior of the particle cloud over time
2D out-of-plane rotation (length of vector yields amount of rotation)
closest
training
template
Dimensionality reduction using Isomap
Let xi be H-dimensional and yi be L-dimensional, where H>L Isomap is a non-linear dimensionality reduction technique that computes a mapping f : xi  yi
Isomap algorithm:
Construct neighborhood graph
Compute shortest paths between points using geodesic distance
Apply classical Multidimensional Scaling (MDS)
Isomap of training data
Geodesic distance – The length of the shortest curve between two points taken along the surface of a manifold. It reflects the true geometry of the manifold.
Top 3 eigenvectors
capture almost 99% of the variation in training data
High-dimensional images reduced to 3 dimensions by Isomap
The blue square indicates the tracker output (weighted average of particles)
Conclusion
Training data
Faces (151 x 151 pixels) with up-down and left-right poses. In-plane rotation is approximated by rotating these templates.
Isomap provides successful framework for pose estimation
Algorithm can handle rapid translation and out-of-plane rotation
Algorithm can track a person and estimate the pose simultaneously
Future goals:
On-line construction of Isomap model so that the particle filter tracker learns as it tracks
Handle occlusions
Block diagram of algorithm for a single particle
Two-dimensional projections of the low-dimensional structure with training templates superimposed