Sampling for Part Based Object Models Daniel Huttenlocher September, 2006

2 Part Based Object Recognition  Matching constellation models, pictorial structures, etc. –Dominated by energy minimization approaches Local or global methods depending on problem definition MAP estimation view  Computationally tractable global optimization depends on models that factor –Appearance of parts independent –Spatial model with low tree width

3 State of the Art?  Model introduced error –Model overly simplistic in order to be tractable  Computationally introduced error –Model “right thing” but don’t know how computational results related  Often not explicit about these sources of error –Precise description of what want to compute and what actually computing

4 Sampling  Statistical method for using tractable (factored) models as means of estimating intractable ones  Proposal distribution –Samples from distribution using factored model evaluated according to more general one –Want “enough” probability mass distributed around in proposal distribution “Promiscuous” – likes multiple things E.g., smoothing a distribution (temperature) –Does more than k-best

5 More Concrete  Part based graphical model, M=(V,E) –Parts V=(v 1, …, v m ) –Spatial relations (undirected edges) E={e ij }  For detection, consider all configurations L P M (I) ≈ max L P M (I|L) P M (L)  Efficient when factors P M (I|L) =  v i V P M (I| l i ) P M (L) =  C  M (L C ) For small cliques C, e.g, 2-cliques for tree

6 A Model that Doesn’t Factor  Patchwork-of-parts (POP) model proposed by Amit and Trouve –Star model with latent reference part –Account for part overlap by averaging probabilities for parts covering an image pixel P M (I|L) no longer factors (sum over parts)  Use likelihood that factors for proposal distribution – overcounting (promiscuous) –Sample from posterior distribution and compute POP probability for these samples Efficiently approximating MAP estimate

7 Sampling Example for Tree [FH05]

8 Comparison with Direct Minimization  Using posterior distribution for factored model – efficient to: –Compute marginals (box sum) –Generate samples For tree, sample location for root from marginal, then sample children conditioned on root location –Evaluate general model on samples  As opposed to trying to optimize general model directly –Using difficult to characterize techniques

9 Simple Experiments  Pictorial structure model using oriented edge part templates  Star topology  Factored appearance model for proposal distribution vs. POP model  Six parts and Caltech-4 data, for comparison with some earlier results using similar models (without POP likelihood) –CFH05, same topology and part models –FPZ05, same topology

10 Detection Results  Single class detection (equal ROC error) –MAP of factored model vs. sampling from factored model –Significant at 95% confidence level except bikes AirplanesCars (rear)FacesMotorbikes MAP (hill climb) 94.3%94.4%98.0%98.6% Sample94.8%95.0%98.4%98.8% CFH05 FPZ % 93.6% 90.3% 84.2% 96.4% 90.3% 93.3% 97.3%

11 Not Limited to Appearance  Sampling is a general technique for approximating intractable distributions –Even easier when using to approximate MAP of those distributions  Tractable distributions can make explicit aspects of problem structure –Over-counting of scene evidence –Importance of kinematic tree spatial constraints for humans, vs. limb coordination