Download presentation

Presentation is loading. Please wait.

Published byJoseph Lade Modified over 2 years ago

1
Interactive Segmentation with Super-Labels Andrew Delong Western Yuri BoykovOlga VekslerLena GorelickFrank Schmidt TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A A A A

2
Natural Images: GMM or MRF? 2 are pixels in this image i.i.d.?NO!

3
Natural Images: GMM or MRF? 3

4
4

5
5

6
Boykov-Jolly / Grab-Cut 6 [Boykov & Jolly, ICCV 2001] [Rother, Kolmogorov, Blake, SIGGRAPH 2004]

7
Boykov-Jolly / Grab-Cut 7 [Boykov & Jolly, ICCV 2001] [Rother, Kolmogorov, Blake, SIGGRAPH 2004]

8
Boykov-Jolly / Grab-Cut 8 [Boykov & Jolly, ICCV 2001] [Rother, Kolmogorov, Blake, SIGGRAPH 2004]

9
Objects within image can be as complex as image itself Where do we draw the line? A Spectrum of Complexity 9 MRF?GMM? Gaussian? object recognition??

10
Single Model Per Class Label 10

11
Multiple Models Per Class Label 11

12
Multiple Models Per Class Label 12

13
Our Energy ¼ Supervised Zhu & Yuille! Zhu & Yuille. PAMI’96; Tu & Zhu. PAMI’02 Unsupervised clustering of pixels 13 boundary length MDL regularizer + color similarity +

14
Our Energy ¼ Supervised Zhu & Yuille! Zhu & Yuille. PAMI’96; Tu & Zhu. PAMI’02 14 boundary length MDL regularizer + color similarity +

15
Interactive Segmentation Example 15

16
Boykov-Jolly / Grab Cut 16 segmentationcolour models

17
Ours 17 segmentationcolour models“sub-labeling”

18
Main Idea Standard MRF: Two-level MRF: 18 object MRF GMMs background MRF image-level MRF object GMMbackground GMM image-level MRF unknown number of labels in each group!

19
The “Super-Pixel” View Complex object ¼ group of super-pixels Interactive segmentation ¼ a“user-constrained super-pixel grouping” 19

20
The “Super-Pixel” View Why not just pre-compute super-pixels? – boundaries may contradict user constraints – user is helpful for making fine distinctions Combine automatic (unsupervised) and interactive (supervised) into single energy 20 Spatially coherent clustering + MDL/complexity penalty + user constraints = 2-level MRF Like Zabih & Kolmogorov, CVPR 2004 Label Costs, CVPR 2010 Like Boykov & Jolly, ICCV 2001

21
Process Overview 21 user constraints propose models from current super-labeling 1 solve 2-level MRF via α-expansion 2 refine all sub-models 3 converged E=503005 E=452288 Boykov-Jolly + unsupervised clustering (random sampling) + iterated multi-label graph cuts (like grab-cut)

22
Our Problem Statement Input: set S of super-labels (e.g. f fg,bg g ) constraints g : P ! S [ f any g 22 fg bg any

23
Our Problem Statement Output: set L of sub-labels sub-labeling f : P ! L model params µ ` for each ` 2L label grouping ¼ : L ! S 23 ¼ ±f¼ ±f f `2`2 `1`1 `3`3 GMM ` 1 white GMM ` 2 dark green light green

24
Our Energy Functional 24 Minimize single energy w.r.t. L, µ, f, ¼ data costssmooth costslabel costs `4`4 `3`3 `1`1 `2`2 forces transition

25
Our Energy Functional 25 Minimize single energy w.r.t. L, µ, f, ¼ data costssmooth costslabel costs pay c 2 `between group’ pay c 1 `within group’

26
Our Energy Functional 26 Minimize single energy w.r.t. L, µ, f, ¼ Penalize number of GMMs used – prefer fewer, simpler models – MDL / information criterion regularize “unsupervised” aspect data costssmooth costslabel costs GMMs

27
More Examples 27 Boykov-Jolly2-level MRF

28
More Examples 28 Boykov-Jolly2-level MRF

29
More Examples 29 Boykov-Jolly 2-level MRF

30
More Examples 30 Boykov-Jolly grad students baby panda 2-level MRF GMM density for blue model

31
Interactive Co-segmentation 31 image collection 2-level MRF Boykov-Jolly (like “iCoseg”, Batra et al., CVPR 2010)

32
More Examples 32 Boykov-Jolly 2-level MRF

33
More Examples 33 Boykov-Jolly 2-level MRF

34
Beyond GMMs 34 GMMs plane GMMs onlyGMMs + planes

35
Synthetic Example 35 GMM Boykov-Jolly (1 GMM each label) GMM 2-level MRF (GMMs only) plane GMM 2-level MRF (GMM + planes) object = two planes in (x,y,grey) space noise = one bi-modal GMM (black;white)

36
Synthetic Example 36 plane GMM black white x 2 planes detected 1 GMM detected y black white

37
As Semi-Supervised Learning Interactive segmentation ¼ a semi-supervised learning – Duchenne, Audibert, Keriven, Ponce, Segonne. Segmentation by Transduction. CVPR 2008. –s - t min cut [Blum & Chawla, ICML’01] – random walker [Szummer & Jaakkola, NIPS’01] 37

38
Conclusions GMM not good enough for image ) GMM not good enough for complex objects Energy-based on 2-level MRF – data costs + smooth costs + label costs Algorithm: iterative random sampling, re-fitting, and ® -expansion. Semi-supervised learning of complex subspaces with ® -expansion 38

Similar presentations

OK

GrabCut Interactive Foreground Extraction using Iterated Graph Cuts Carsten Rother Vladimir Kolmogorov Andrew Blake Microsoft Research Cambridge-UK.

GrabCut Interactive Foreground Extraction using Iterated Graph Cuts Carsten Rother Vladimir Kolmogorov Andrew Blake Microsoft Research Cambridge-UK.

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google