Presentation on theme: "Interactive Segmentation with Super-Labels Andrew Delong Western Yuri BoykovOlga VekslerLena GorelickFrank Schmidt TexPoint fonts used in EMF. Read the."— Presentation transcript:
Interactive Segmentation with Super-Labels Andrew Delong Western Yuri BoykovOlga VekslerLena GorelickFrank Schmidt TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A A A A
Natural Images: GMM or MRF? 2 are pixels in this image i.i.d.?NO!
Main Idea Standard MRF: Two-level MRF: 18 object MRF GMMs background MRF image-level MRF object GMMbackground GMM image-level MRF unknown number of labels in each group!
The “Super-Pixel” View Complex object ¼ group of super-pixels Interactive segmentation ¼ a“user-constrained super-pixel grouping” 19
The “Super-Pixel” View Why not just pre-compute super-pixels? – boundaries may contradict user constraints – user is helpful for making fine distinctions Combine automatic (unsupervised) and interactive (supervised) into single energy 20 Spatially coherent clustering + MDL/complexity penalty + user constraints = 2-level MRF Like Zabih & Kolmogorov, CVPR 2004 Label Costs, CVPR 2010 Like Boykov & Jolly, ICCV 2001
Process Overview 21 user constraints propose models from current super-labeling 1 solve 2-level MRF via α-expansion 2 refine all sub-models 3 converged E=503005 E=452288 Boykov-Jolly + unsupervised clustering (random sampling) + iterated multi-label graph cuts (like grab-cut)
Our Problem Statement Input: set S of super-labels (e.g. f fg,bg g ) constraints g : P ! S [ f any g 22 fg bg any
Our Problem Statement Output: set L of sub-labels sub-labeling f : P ! L model params µ ` for each ` 2L label grouping ¼ : L ! S 23 ¼ ±f¼ ±f f `2`2 `1`1 `3`3 GMM ` 1 white GMM ` 2 dark green light green
Our Energy Functional 24 Minimize single energy w.r.t. L, µ, f, ¼ data costssmooth costslabel costs `4`4 `3`3 `1`1 `2`2 forces transition
Our Energy Functional 25 Minimize single energy w.r.t. L, µ, f, ¼ data costssmooth costslabel costs pay c 2 `between group’ pay c 1 `within group’
Our Energy Functional 26 Minimize single energy w.r.t. L, µ, f, ¼ Penalize number of GMMs used – prefer fewer, simpler models – MDL / information criterion regularize “unsupervised” aspect data costssmooth costslabel costs GMMs
Synthetic Example 35 GMM Boykov-Jolly (1 GMM each label) GMM 2-level MRF (GMMs only) plane GMM 2-level MRF (GMM + planes) object = two planes in (x,y,grey) space noise = one bi-modal GMM (black;white)
Synthetic Example 36 plane GMM black white x 2 planes detected 1 GMM detected y black white
As Semi-Supervised Learning Interactive segmentation ¼ a semi-supervised learning – Duchenne, Audibert, Keriven, Ponce, Segonne. Segmentation by Transduction. CVPR 2008. –s - t min cut [Blum & Chawla, ICML’01] – random walker [Szummer & Jaakkola, NIPS’01] 37
Conclusions GMM not good enough for image ) GMM not good enough for complex objects Energy-based on 2-level MRF – data costs + smooth costs + label costs Algorithm: iterative random sampling, re-fitting, and ® -expansion. Semi-supervised learning of complex subspaces with ® -expansion 38