1. Mean-shift and its application for object tracking
Dorin Comaniciu, Visvanathan Ramesh and Peter Meer “Kernel-Based Object Tracking”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.25, No 5, May 2003
Dorin Comaniciu and Peter Meer, “Mean shift: a robust approach toward feature space analysis”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.25, no 5, May 2002
Dorin Comaniciu and Peter Meer, “Mean shift analysis and applications", The Proceedings of the Seventh IEEE International Conference on Computer Vision, vol.2, pp , September 1999
Andy

2. Density GRADIENT Estimation
What is mean-shift?
A tool for: finding modes in a set of data samples, manifesting an underlying probability density function (PDF) in RN
Non-parametric
Density Estimation
Data
Discrete PDF Representation
Non-parametric
Density GRADIENT Estimation
(Mean Shift)
PDF Analysis

3. Intuitive description
Objective : Find the densest region
Region of
interest
Center of
mass
Mean Shift
vector

4. Intuitive description
Objective : Find the densest region
Region of
interest
Center of
mass
Mean Shift
vector

5. Intuitive description
Objective : Find the densest region
Region of
interest
Center of
mass
Mean Shift
vector

6. Intuitive description
Objective : Find the densest region
Region of
interest
Center of
mass
Mean Shift
vector

7. Intuitive description
Objective : Find the densest region
Region of
interest
Center of
mass
Mean Shift
vector

8. Intuitive description
Objective : Find the densest region
Region of
interest
Center of
mass
Mean Shift
vector

9. Intuitive description
Objective : Find the densest region
Region of
interest
Center of
mass

10. Modality analysis Tessellate the space with windows
Run the procedure in parallel

11. Modality analysis
The blue data points were traversed by the windows towards the mode

12. Modality analysis
Window tracks signify the steepest ascent directions

13. Mean shift for data clustering

14. Data clustering example
Simple Modal Structures
Complex Modal Structures

15. Data clustering example
Feature space:
L*u*v representation
Initial window
centers
Modes found
Modes after
pruning
Final clusters

16. Image segmentation examples

17. General framework: target representation
Choose a reference model in the current frame
Choose a feature space
Represent the model in the chosen feature space
Current frame …

18. General framework: target localization
Start from the position of the model in the current frame
Search in the model’s neighborhood in next frame
Find best candidate by maximizing a similarity func.
Repeat the same process in the next pair of frames
Current frame …
Model
Candidate

19. Target representation
Choose a reference target model
Represent the model by its PDF in the feature space
Quantized Color Space
Choose a feature space

20. PDF representation Target Model Target Candidate Similarity Function:
(centered at 0)
Target Candidate
(centered at y)
Similarity Function:

21. Smoothness of the similarity function
Problem:
Target is represented by color info only
Spatial info is lost
Large similarity variations for adjacent locations
f is not smooth
Gradient-based optimizations are not robust
Solution:
Mask the target with an isotropic kernel in the spatial domain
f(y) becomes smooth in y

22. PDF with spatial weighting
model y
candidate
Target pixel locations
A differentiable, isotropic, convex, monotonically decreasing kernel
Peripheral pixels are affected by occlusion and background interference
The color bin index (1..m) of pixel x
Probability of feature u in model
Probability of feature u in candidate
Normalization factor
Pixel weight
Normalization factor
Pixel weight

23. The Bhattacharyya Coefficient
Similarity function
Target model:
Target candidate:
Similarity function:
The Bhattacharyya Coefficient 1

24. Target localization algorithm
Start from the position of the model in the current frame
Search in the model’s neighborhood in next frame
Find best candidate by maximizing a similarity func.

25. Approximating similarity function
Model location:
Candidate location:
Linear approx.
(around y0)
Independent of y
Density estimation (as a function of y)

26. Computing gradient of similarity function
Simple Mean Shift procedure:
Compute mean shift vector
Translate the Kernel window by m(x)

27. Kernels and kernel profiles
k – kernel profile
K– kernel
Epanechnikov Kernel
Uniform Kernel
Normal Kernel

28. Epanechnikov kernel A special class of radially symmetric kernels:
Uniform kernel:

29. Tracking with mean-shift
1. Initialize the location of the target in the current frame (y0) with location of model on previous frame, compute candidate target model p(y0) and evaluate

2. Compute weights
3. Find the next location of the target candidate according to
4. Compute candidate model at new position p(y1)
5. If ||y0-y1||<ε Stop.
Else – set y0 = y1 and go to step 2.

30. Conclusions Advantages : Disadvantages:
Application independent tool
Suitable for real data analysis
Does not assume any prior shape (e.g. elliptical) on data clusters
Can handle arbitrary feature spaces
Only ONE parameter to choose
h (window size) has a physical meaning, unlike K-Means
Disadvantages:
The window size (bandwidth selection) is not trivial
Inappropriate window size can cause modes to be merged, or generate additional “shallow” modes  Use adaptive window size