Download presentation

Presentation is loading. Please wait.

Published byJoseph Lade Modified over 2 years ago

1
Interactive Segmentation with Super-Labels Andrew Delong Western Yuri BoykovOlga VekslerLena GorelickFrank Schmidt TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A A A A

2
Natural Images: GMM or MRF? 2 are pixels in this image i.i.d.?NO!

3
Natural Images: GMM or MRF? 3

4
4

5
5

6
Boykov-Jolly / Grab-Cut 6 [Boykov & Jolly, ICCV 2001] [Rother, Kolmogorov, Blake, SIGGRAPH 2004]

7
Boykov-Jolly / Grab-Cut 7 [Boykov & Jolly, ICCV 2001] [Rother, Kolmogorov, Blake, SIGGRAPH 2004]

8
Boykov-Jolly / Grab-Cut 8 [Boykov & Jolly, ICCV 2001] [Rother, Kolmogorov, Blake, SIGGRAPH 2004]

9
Objects within image can be as complex as image itself Where do we draw the line? A Spectrum of Complexity 9 MRF?GMM? Gaussian? object recognition??

10
Single Model Per Class Label 10

11
Multiple Models Per Class Label 11

12
Multiple Models Per Class Label 12

13
Our Energy ¼ Supervised Zhu & Yuille! Zhu & Yuille. PAMI’96; Tu & Zhu. PAMI’02 Unsupervised clustering of pixels 13 boundary length MDL regularizer + color similarity +

14
Our Energy ¼ Supervised Zhu & Yuille! Zhu & Yuille. PAMI’96; Tu & Zhu. PAMI’02 14 boundary length MDL regularizer + color similarity +

15
Interactive Segmentation Example 15

16
Boykov-Jolly / Grab Cut 16 segmentationcolour models

17
Ours 17 segmentationcolour models“sub-labeling”

18
Main Idea Standard MRF: Two-level MRF: 18 object MRF GMMs background MRF image-level MRF object GMMbackground GMM image-level MRF unknown number of labels in each group!

19
The “Super-Pixel” View Complex object ¼ group of super-pixels Interactive segmentation ¼ a“user-constrained super-pixel grouping” 19

20
The “Super-Pixel” View Why not just pre-compute super-pixels? – boundaries may contradict user constraints – user is helpful for making fine distinctions Combine automatic (unsupervised) and interactive (supervised) into single energy 20 Spatially coherent clustering + MDL/complexity penalty + user constraints = 2-level MRF Like Zabih & Kolmogorov, CVPR 2004 Label Costs, CVPR 2010 Like Boykov & Jolly, ICCV 2001

21
Process Overview 21 user constraints propose models from current super-labeling 1 solve 2-level MRF via α-expansion 2 refine all sub-models 3 converged E=503005 E=452288 Boykov-Jolly + unsupervised clustering (random sampling) + iterated multi-label graph cuts (like grab-cut)

22
Our Problem Statement Input: set S of super-labels (e.g. f fg,bg g ) constraints g : P ! S [ f any g 22 fg bg any

23
Our Problem Statement Output: set L of sub-labels sub-labeling f : P ! L model params µ ` for each ` 2L label grouping ¼ : L ! S 23 ¼ ±f¼ ±f f `2`2 `1`1 `3`3 GMM ` 1 white GMM ` 2 dark green light green

24
Our Energy Functional 24 Minimize single energy w.r.t. L, µ, f, ¼ data costssmooth costslabel costs `4`4 `3`3 `1`1 `2`2 forces transition

25
Our Energy Functional 25 Minimize single energy w.r.t. L, µ, f, ¼ data costssmooth costslabel costs pay c 2 `between group’ pay c 1 `within group’

26
Our Energy Functional 26 Minimize single energy w.r.t. L, µ, f, ¼ Penalize number of GMMs used – prefer fewer, simpler models – MDL / information criterion regularize “unsupervised” aspect data costssmooth costslabel costs GMMs

27
More Examples 27 Boykov-Jolly2-level MRF

28
More Examples 28 Boykov-Jolly2-level MRF

29
More Examples 29 Boykov-Jolly 2-level MRF

30
More Examples 30 Boykov-Jolly grad students baby panda 2-level MRF GMM density for blue model

31
Interactive Co-segmentation 31 image collection 2-level MRF Boykov-Jolly (like “iCoseg”, Batra et al., CVPR 2010)

32
More Examples 32 Boykov-Jolly 2-level MRF

33
More Examples 33 Boykov-Jolly 2-level MRF

34
Beyond GMMs 34 GMMs plane GMMs onlyGMMs + planes

35
Synthetic Example 35 GMM Boykov-Jolly (1 GMM each label) GMM 2-level MRF (GMMs only) plane GMM 2-level MRF (GMM + planes) object = two planes in (x,y,grey) space noise = one bi-modal GMM (black;white)

36
Synthetic Example 36 plane GMM black white x 2 planes detected 1 GMM detected y black white

37
As Semi-Supervised Learning Interactive segmentation ¼ a semi-supervised learning – Duchenne, Audibert, Keriven, Ponce, Segonne. Segmentation by Transduction. CVPR 2008. –s - t min cut [Blum & Chawla, ICML’01] – random walker [Szummer & Jaakkola, NIPS’01] 37

38
Conclusions GMM not good enough for image ) GMM not good enough for complex objects Energy-based on 2-level MRF – data costs + smooth costs + label costs Algorithm: iterative random sampling, re-fitting, and ® -expansion. Semi-supervised learning of complex subspaces with ® -expansion 38

Similar presentations

OK

A. Criminisi, T. Sharp and K. Siddiqui. Properties of our algorithm efficient on high-res./nD images (~milliseconds) easy to edit and fix accurate (e.g.

A. Criminisi, T. Sharp and K. Siddiqui. Properties of our algorithm efficient on high-res./nD images (~milliseconds) easy to edit and fix accurate (e.g.

© 2018 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on acid-base indicators you can make at home Ppt on value of pie in math Ppt on eisenmenger syndrome treatment Ppt on idioms and phrases Download ppt on print culture and modern world Ppt on web design and development Ppt on bluetooth free download Ppt on marine ecosystem Ppt on pricing policy template Ppt on nonverbal communication body language