1. Dynamical Statistical Shape Priors for Level Set Based Tracking
Dheeraj Singaraju
Reading group: 07/27/06

2. Segmentation using shape priors
In the past, level set based segmentation methods have incorporated statistical shape knowledge
Statistically learned shape information can deal with missing information or misleading information due to noise, clutter or occlusion
Such segmentation frameworks can be applied to tracking objects across frames

3. The need for dynamic shape priors
All the previous approaches in tracking assume statistical shape priors that are static in time
In silhouette data corresponding to a walking person, certain poses become less / more likely with time
We need to exploit the temporal coherence in the poses of a human during the course of an entire action

4. Aim of the paper
This paper deals with learning dynamical statistical shape models for implicitly represented shapes to track human motion
Such an approach segments an image based on its intensities and the segmentations obtained in the previous image frames
The resulting approach is expected to perform better in the presence of noise and clutter than the previous mentioned approaches, due to the temporal shape consistency being enforced.

5. Features of the proposed algorithm
Implicit shape representation: The method can deal with shapes of varying topology
Intensity based segmentation: The tracking scheme is region based as opposed to being edge based
Optimization using gradient descent: Facilitates an extension to data in higher dimension

6. Implicit representation of hyperspaces
The level set method helps propagate hyperspaces in a domain by evolving an appropriate embedding function , where
Advantages of such a representation
The boundary representation does not depend on any parameterization
Topological changes such as merging and splitting can be dealt with
One can generalize the framework easily to a higher dimensional data

7. Defining terms to work with
Shape( ): A set of closed 2D contours modulo a certain transformation
Transformations( ): They are essentially problem dependent and can be rigid body transformations, similarity or affine transforms, etc
The object of interest is thus
We want to model the temporal evolution of the shapes separately from that of the transformations

8. The Bayesian inference problem
Assume we are given consecutive images from an image sequence where denotes the set of images
We then want to maximize the following conditional probability
Does not depend on the estimated quantities
temporal consistency
goodness of segmentation

9. Assumptions to make life easier
The images are assumed to be mutually independent
The intensities of the shape and the background are modeled as independent samples from two Gaussian distributions with unknown means and unknown variances

10. Evaluating the Gaussian models
We use the Heaviside step function to denote the areas where is positive or negative

11. Simplification of distribution
To avoid computational burden, it is assumed that the distribution of previous states of and to be strongly peaked around the maxima of the respective distribution
If we assume that the tracking system does not stored the previous images but only the past estimates of shape and transformation, then the inference problem reduces to

12. Distributions for temporal evolutions
We now need to evaluate
We break down the analysis into two cases
We assume that the shape and transformation are mutually independent and assume a uniform prior on the transformation parameters
We consider the joint distribution of the shape and transformation parameters

13. Shapes and eigenmodes
It is known that statistical models can be estimated more reliably if the dimensionality of the model and data are low
The Bayesian inference problem is now formulated in low dimensions within the subspace spanned by the largest principal eigenmodes of a set of a set of sample shapes
The training sequence is therefore used for the following
Extraction of eigenmodes of the sample shapes
Learning of dynamical models for the low dimensional representation of the implicit shapes

14. Projection into the space of shapes
If is a temporal sequence of training shapes. We denote the mean shape as and the n most significant eigenmodes as
Given the mean and n most significant eigenmodes, we can represent any arbitrary shape by a shape vector as follows

15. How effective is the projection ?
Actual walking sequence
Approximated walking sequence using 6 eigenmodes

16. Projections instead of the actual shape
Now we work with the parameters which are believed to be synonymous with the segmentations
We therefore need to maximize the following conditional probability
Contribution of this paper

17. Temporal evolution of shapes
The paper proposes to learn the temporal dynamics by modeling the shape vectors by a Markov chain of order k
The probability of a shape conditioned on the shapes at the previous times is then modeled as
DOUBT
Dependent on time ?

18. Effectiveness of predicting evolution
The paper considers 151 frames of a training sequence and estimates the parameters of a second order autoregressive model
These model parameters can then be used to synthesize walking sequences

19. Synthesis of walking sequence
The synthesized sequences captures most of the characteristic motion of the walking person
Discrepancies in the shapes are due to a great reduction in the number of parameters used to describe the motion

20. Coupling the shape and transformation parameters
In general, one can expect the deformation parameters and the transformation parameters to be coupled
We want to learn dynamical models that are invariant to rotation, translation or any other desired transformations
Instead of the absolute transformation, we consider the incremental transformation between frames
We deal with a new vector as rather than a shape vector

21. Optimization using gradient descent
Given an image and a set of previously segmented shape parameters and transformation parameters the goal is to maximize the conditional probability with respect to
This goal can be achieved by minimizing the following energy function
Relative weighting between the shape prior and the data term

22. A close look at the energy function
The data term can be written as
The shape term can be written as

23. Updating the shape
Update equation with respect to the shape parameters
separation of image intensities
draws the shape towards a prior

24. Updating the transformation parms
Update equation with respect to the transformation parms
draws the shape towards the most likely transformation
draws the shape towards a prior

25. Noisy images of a man walking
Results
Noisy images of a man walking
Segmentation using static shape priors, 25%noise
Segmentation using static shape priors, 50%noise

26. Results (contd.) Segmentation using dynamic shape priors, 50% noise

27. Segmentation using dynamic shape priors, 75% noise
Results (contd.)
Segmentation using dynamic shape priors, 75% noise

28. Results (contd.)

29. Results (contd.)
Comparison of algorithm with ground truth

30. Results (contd.) Invariance to speed
To increase speed – removed frames
To decrease speed – replicated and inserted frames