TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical University, Prague, 3K.U. Leuven, 4University of Basel

Outline  1. Introduction  2. Learning the Motion Couplings  3. Practical Implementation  4. Experimental Results  5. Conclusion

Outline  1. Introduction

Introduction  Goal: Estimate the Position of an Object

Introduction  Many temporary, but potential very strong links exists between a tracked object and parts of the images.  We discover the dynamic elements – called SUPPORTERS – that predict the position of the target. [L. Cerman, J. Matas, J., V. Hlavac, Sputnik Tracker,SCIA, 2009] [M. Yang, Y. Wu, G. Hua. Context-Aware Visual Tracking PAMI, 2009]

SUPPORTERS  1. Came with different strength.  2. Change over time.  SUPPORTERS help Tracking of objects which change there appearance very quickly and occluded objects or object outside the image.

Tracking Carl  Given the position of the object of interest in the frame.  Local image features from the whole image are extracted (yellow points). Image features: our implementation uses Harris points [5] and describe them by a SIFT inspired descriptor [10].

Local Image Features as Supporters  These image features are usually divided into object points and points belonging to the background.

Local Image Features as Supporters  Object points lie on the object surface and thus always have a strong correlation to the object motion (green points)

Local Image Features as Supporters  The object gets occluded or changed its appearance, the supporters can help infer its position.

Discovering the Supporters

Outline  2. Learning the Motion Couplings

Problem Formulation  We want to learn a model of P(x|I), predicting the position x of an object in the image I.

Implicit Shape Model - Features  After training from a large database of labeled images, the model can be used to detect objects from that object class in test images.  indicator functions P(f|I)

Implicit Shape Model – Object Displacement Each feature subsequently casts probabilistic votes for possible object positions, where the hypothesis score is obtained as a sum over all votes. P(x|f):votes for the object position x.

Implicit Shape Model

Model of Carl – Object Detection

Model of Supporters

Voting Where μ is the mean and Σ is the covariance matrix.

Reliable Information & Motion Coupling feature point x(i) = (x(i), y(i)) associated with a feature f(i) and the position x* = (x*, y*) of the target.  where parameter α ϵ [0, 1] determines the weighting of the past estimates.  The current image It and the object position Xt* is included using p(f|It),  the indicator of feature f in the current image;  and p(Xt*-Xt(i)|f), the current relative object position.

Implementation angle Ф distance r covariance matrix Σ S tore all detected feature points in a database DB. The entries (d,r, Ф, Σ ) store the descriptor and the estimated voting vector, encoded by radius, angle and covariance matrix.

Updates Adjust the voting vector, the radius r and the relative angle Ф Update the covariance matrix Σ

Outline  3. Practical Implementation

Algorithm : Determination of Position

Outline  4. Experimental Results

ETH-Cup: Supporters

[11] S. Stalder, H. Grabner, and L. V. Gool. Beyond semisupervised tracking: Tracking should be as simple as detection,but not simpler than recognition. In Proc. IEEE WS on On-line Learning for Computer Vision, 2009.

Experimental Results

ETH-Cup: Improving Object Tracking   Recall: TP/(TP+FN)  Precision: TP/(TP+FP) 

Medical Application  (a) Manually labeled valve positions, (b-f) the found valve positions in 5 cardiac phases. As the confidence of the voting space was too low, in cardiac phase 16, the valve positions were interpolated (indicated by the dotted lines) from neighboring images.

Outline  5. Conclusion

Conclusion  SUPPORTERS help Tracking of objects which 1.Change there appearance very quickly. 2.Occluded objects or object outside the image.