12 th International Fall Workshop VISION, MODELING, AND VISUALIZATION 2007 November 7-9, 2007 Saarbrücken, Germany Estimating Natural Activity by Fitting 3D Models via Learned Objective Functions 1 Faculty of Science and Engineering, Waseda University, Tokyo , Japan 2 Institut für Informatik, Technische Universität München, Garching, Germany 3 Kognitive Neuroinformatik, Universität Bremen, Bremen, Germany Matthias Wimmer 1, Christoph Mayer 2, Freek Stulp 3 and Bernd Radig 2

Natural Activity tactile channel visual channel auditory channel olfactory channel auditory channel visual channel tactile channel olfactory channel

Model-based Image Interpretation Model Describes the image content with the help of a parameter vector p. Objective Function Calculates how well a parameterized model p fits to an image I. Fitting Algorithm Optimizes the objective function and therefore estimates the model that fits the image best.

Objective Functions f(I,p) Splitting the Objective Function to Local Objective Functions Evaluate one objective function per model point. Approximate the model parameters. Evaluation of the Objective Function Along characteristic direction. Often perpendicular to the model.

Traditional Approach Shortcomings:  Requires domain knowledge.  Based on designer’s intuition.  Time-consuming. Manually design the objective function Manually evaluate on test images designed objective function good not good

Ideal Objective Functions P1:Correctness Property: The global minimum corresponds to the best model fit. P2:Uni-modality Property: The objective function has no local extrema. ¬ P1 P1 ¬P2 P2

Learning the Objective Function (1)

Learning the Objective Function (2) Generate further Annotations Annotations in three-dimensional space along characteristic direction One characteristic direction is not sufficient.

Learning the Objective Function (3)

Learning the Objective Function (4) Three characteristic directions in three-dimensional space. Model point is moved along the most important characteristic direction. Characteristic direction with largest angle to the image normal is considered most important.

Learning the Objective Function (5)

Learning the Objective Function (6) Advantages  The loop is removed.  The objective function approximates the ideal objective function.  No domain-dependent knowledge is needed.

Evaluation General approach Uniformly distributed error is applied to models and fitting is performed afterwards. Distances are measured in centimeters. Fraction of models located at a certain distance or better is evaluated. Two objective functions f A and f B with learning radii ∆ A = 3 × ∆ B.

Evaluation (2) f B handles small displacements better. f A handles large displacements better. Subsequent execution shows both advantages.

Evaluation (3) Results are improved with every iteration. Lower bound of quality is reached after several iterations.

Thank you !