Recovering Articulated Object Models from 3D Range Data Dragomir Anguelov Daphne Koller Hoi-Cheung Pang Praveen Srinivasan Sebastian Thrun Computer Science.

Slides:



Advertisements
Similar presentations
The Primal-Dual Method: Steiner Forest TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A AA A A A AA A A.
Advertisements

O BJ C UT M. Pawan Kumar Philip Torr Andrew Zisserman UNIVERSITY OF OXFORD.
Active Appearance Models
Structured SVM Chen-Tse Tsai and Siddharth Gupta.
Support Vector Machines
Online Social Networks and Media. Graph partitioning The general problem – Input: a graph G=(V,E) edge (u,v) denotes similarity between u and v weighted.
Robust Global Registration Natasha Gelfand Niloy Mitra Leonidas Guibas Helmut Pottmann.
Modeling the Shape of People from 3D Range Scans
Model base human pose tracking. Papers Real-Time Human Pose Tracking from Range Data Simultaneous Shape and Pose Adaption of Articulated Models using.
Proportion Priors for Image Sequence Segmentation Claudia Nieuwenhuis, etc. ICCV 2013 Oral.
1 Fast Primal-Dual Strategies for MRF Optimization (Fast PD) Robot Perception Lab Taha Hamedani Aug 2014.
A Probabilistic Framework for Semi-Supervised Clustering
Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.
Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,
Reverse Engineering Niloy J. Mitra.
One-Shot Multi-Set Non-rigid Feature-Spatial Matching
Support Vector Machines (SVMs) Chapter 5 (Duda et al.)
Locally Constraint Support Vector Clustering
Iterative closest point algorithms
Solving Integer Programs. Natural solution ideas that don’t work well Solution idea #1: Explicit enumeration: Try all possible solutions and pick the.
Prénom Nom Document Analysis: Linear Discrimination Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.
A Study of Approaches for Object Recognition
Correspondence & Symmetry
2010/5/171 Overview of graph cuts. 2010/5/172 Outline Introduction S-t Graph cuts Extension to multi-label problems Compare simulated annealing and alpha-
Prénom Nom Document Analysis: Data Analysis and Clustering Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.
1 Numerical geometry of non-rigid shapes Spectral Methods Tutorial. Spectral Methods Tutorial 6 © Maks Ovsjanikov tosca.cs.technion.ac.il/book Numerical.
Tracking Video Objects in Cluttered Background
Multiagent Planning with Factored MDPs Carlos Guestrin Daphne Koller Stanford University Ronald Parr Duke University.
Clustering Vertices of 3D Animated Meshes
Radial Basis Function Networks
3D Global Registration. The Problem Given: n scans around an objectGiven: n scans around an object Goal: align them allGoal: align them all First attempt:
Domain decomposition in parallel computing Ashok Srinivasan Florida State University COT 5410 – Spring 2004.
Exploiting Segmentation for Robust 3D Object Matching Michael Krainin, Kurt Konolige, and Dieter Fox.
CSE554Laplacian DeformationSlide 1 CSE 554 Lecture 8: Laplacian Deformation Fall 2012.
Stochastic k-Neighborhood Selection for Supervised and Unsupervised Learning University of Toronto Machine Learning Seminar Feb 21, 2013 Kevin SwerskyIlya.
1/20 Obtaining Shape from Scanning Electron Microscope Using Hopfield Neural Network Yuji Iwahori 1, Haruki Kawanaka 1, Shinji Fukui 2 and Kenji Funahashi.
Niloy J. Mitra Leonidas J. Guibas Mark Pauly TU Vienna Stanford University ETH Zurich SIGGRAPH 2007.
MRFs and Segmentation with Graph Cuts Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/24/10.
ALIGNMENT OF 3D ARTICULATE SHAPES. Articulated registration Input: Two or more 3d point clouds (possibly with connectivity information) of an articulated.
Chapter 14: SEGMENTATION BY CLUSTERING 1. 2 Outline Introduction Human Vision & Gestalt Properties Applications – Background Subtraction – Shot Boundary.
CSC321: Neural Networks Lecture 12: Clustering Geoffrey Hinton.
Object Stereo- Joint Stereo Matching and Object Segmentation Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on Michael Bleyer Vienna.
Learning a Small Mixture of Trees M. Pawan Kumar Daphne Koller Aim: To efficiently learn a.
Generalized Hough Transform
Linear Programming Erasmus Mobility Program (24Apr2012) Pollack Mihály Engineering Faculty (PMMK) University of Pécs João Miranda
AS-RIGID-AS-POSSIBLE SHAPE MANIPULATION
Temporally Coherent Completion of Dynamic Shapes AUTHORS:HAO LI,LINJIE LUO,DANIEL VLASIC PIETER PEERS,JOVAN POPOVIC,MARK PAULY,SZYMON RUSINKIEWICZ Presenter:Zoomin(Zhuming)
Probabilistic Graphical Models seminar 15/16 ( ) Haim Kaplan Tel Aviv University.
Radial Basis Function ANN, an alternative to back propagation, uses clustering of examples in the training set.
SCAPE: Shape Completion and Animation PEople Stanford University Dragomir Anguelov Praveen Srinivasan Daphne Koller Sebastian Thrun Jim Rodgers UC, Santa.
Relation Strength-Aware Clustering of Heterogeneous Information Networks with Incomplete Attributes ∗ Source: VLDB.
A global approach Finding correspondence between a pair of epipolar lines for all pixels simultaneously Local method: no guarantee we will have one to.
Affine Registration in R m 5. The matching function allows to define tentative correspondences and a RANSAC-like algorithm can be used to estimate the.
Markov Networks: Theory and Applications Ying Wu Electrical Engineering and Computer Science Northwestern University Evanston, IL 60208
Photoconsistency constraint C2 q C1 p l = 2 l = 3 Depth labels If this 3D point is visible in both cameras, pixels p and q should have similar intensities.
Energy minimization Another global approach to improve quality of correspondences Assumption: disparities vary (mostly) smoothly Minimize energy function:
Deformation Modeling for Robust 3D Face Matching Xioguang Lu and Anil K. Jain Dept. of Computer Science & Engineering Michigan State University.
Semi-Supervised Clustering
Data Driven Resource Allocation for Distributed Learning
Gonçalo Abecasis and Janis Wigginton University of Michigan, Ann Arbor
The minimum cost flow problem
Morphing and Shape Processing
You can check broken videos in this slide here :
Efficiently Selecting Regions for Scene Understanding
Chapter 6. Large Scale Optimization
Integer Programming (정수계획법)
CSE 554 Lecture 10: Extrinsic Deformations
Integer Programming (정수계획법)
Text Categorization Berlin Chen 2003 Reference:
Chapter 6. Large Scale Optimization
Presentation transcript:

Recovering Articulated Object Models from 3D Range Data Dragomir Anguelov Daphne Koller Hoi-Cheung Pang Praveen Srinivasan Sebastian Thrun Computer Science Dept. Stanford University

Problem overview Data: Articulated object observed in different configurations 3D range surfaces (meshes) obtained by a scanner Task: Recover the articulated object model 1. Register all surfaces 2. Cluster the object surface into rigid parts 3. Recover joints linking the parts Applications of articulated models Animation, object detection, scene segmentation

Surface registration Algorithm input: meshes Z 0, Z 1, … Z N Pick template mesh X = Z 0 Need to recover point-to-point correspondences between X and instance meshes Z i, i={1, …, N} Any registration algorithm providing can be used Correlated Correspondences algorithm [Anguelov et al ‘04] Unsupervised nonrigid registration algorithm Handles large deformations, occlusion (but not clutter) Algorithm Overview Computes embedding of mesh Z into mesh X defined as a joint assignment to correspondences Encodes correspondence correlations (minimizing deformation, preserving geodesic distances) as pairwise potentials in a Markov network over Uses Loopy-BP to obtain a good joint assignment to all variables 22 11 33

Clustering into rigid parts: Generative model Template X Transformed Model Instance Z Transformations z1z1 b1b1 a1a1 Tx1x1 y1y1 aMaM xMxM yMyM zkzk bkbk … … … … Part labels Points Point labels Points  j = Part (x j )  k = ModelPoint (z k ) Assume  values provided by a nonrigid registration algorithm 1. Rigid part points share the same rigid transformaition in each instance rotationtranslation 2. Each part is a connected region of the template mesh surface a) Soft contiguity constraints: If and are connected in X for b) Hard contiguity constraints: Each part consists of a single connected component of the surface of X

Clustering into rigid parts: Optimization Model Summary: Ignoring hard contiguity constraints, generative model defines a Markov Network over the part labels : Objective Iterative Optimization E-step: Given transformations, compute the part labels maximizing Eq. (1) and satisfying the hard contiguity constraints M-step: Given part labels, compute transformations Point in mesh Z i, generated by template point x j

Clustering into rigid parts: E-Step The problem of finding a MAP assignment to given which maximizes Eq. (1) can be expressed as an integer program (IP). A relaxation of the IP produces a linear program (LP) [Kleinberg et al ’99]. For 2-part models, the optimal LP solution is also the optimal IP solution. For n-part models, n>2: In case of a fractional LP solution, a rounding procedure produces an IP solution within a factor of 2 of the optimal. The LP can be solved very efficiently by using multiway min-cuts [Kolmogorov et al ’04] The LP solution may not satisfy hard contiguity constraints. A part consisting of m connected components is split into m separate parts which doesn’t change the objective in Eq. (1).

Clustering into rigid parts: LP formulation Introduce variables s.t. Log-cost associated with potential becomes where Define separation cost of an edge as where Putting everything together: Relax binary constraints to obtain LP

Clustering into rigid parts: M-step and Annealing M-step: find rigid part transformations given part labels Optimization function decomposes into a separate equation for each : Eq. (2) is isomorphic to the ICP problem [Besl’ 92]. Close form estimate of is obtained by solving a simple LP. Annealing Number of parts in solution depends on the tradeoff where large values of result in fewer parts When estimate of is poor and is large, solving E-step LP is hard, because multiple plausible solutions with similar scores exist Starting with low values of and gradually increasing performs best in practice

Clustering into Rigid Parts: Initialization Need good initial values for the transformations Template X Method 1: Cluster local rigid transformations [Cheung et al ’03] Associate a vector of transformations with each point (estimate using the known correspondences for all instance meshes) Cluster points into k parts using adaptive PCA [Archer et al ’02] Slow: >10 mins for 2000 point models Method 2: Break surface into k arbitrary fragments of approximately equal size Fast: <1s for 2000 point models Same final results! a) Rigid parts after initialization b) Rigid parts after EM a) b)

Optimizing the Joint Positions A joint is a point belonging to two parts simultaneously: [Cheung et al ’03] To solve for the joint, must optimize Sometimes, the solution can be an entire space of points Example: a joint with one degree of freedom Introduce an additional regularization term, enforcing that the joint is close to the boundary between two parts in X a) b) a) Rigid parts after EM b) Estimated joints Joint between parts p and q

Results: Puppet Dataset Correctly identifies all 15 puppet parts and the skeleton

Results: Initialization Testing EM Initialization Initialize with a different number of parts k If k > 15, the algorithm consistently finds the correct number of parts (15)

Results: Arm Dataset Correctly identifies 4 arm parts and the skeleton in the presence of non-rigid deformation

Results: Annealing Results of annealing with increasing value of Increasing decreases the number of parts (A – D) Increasing too much results in a partition which tries to reduce the number of links between parts (E)

Conclusions and Future Work Algorithm for recovering articulate object models from 3D data Clusters the object surface into rigid parts Recovers the articulated object skeleton Algorithm properties Probabilistic formulation Efficient global inference Algorithm recovers complex models consisting of a large number of parts in real world datasets Future work Extend the algorithm to allow and explicitly model (small) deformations of the articulated parts Robustness to local maxima