Presentation is loading. Please wait.

Presentation is loading. Please wait.

Adaptive Segmentation Based on a Learned Quality Metric I. Frosio 1, E. Ratner 2 1 NVIDIA, USA, 2 Lyrical Labs, USA.

Similar presentations


Presentation on theme: "Adaptive Segmentation Based on a Learned Quality Metric I. Frosio 1, E. Ratner 2 1 NVIDIA, USA, 2 Lyrical Labs, USA."— Presentation transcript:

1 Adaptive Segmentation Based on a Learned Quality Metric I. Frosio 1, E. Ratner 2 1 NVIDIA, USA, 2 Lyrical Labs, USA

2 2 Motivation: good / bad segmentation SLIC (Achanta, 2012)

3 3 Motivation: good / bad segmentation GRAPH-CUT (Felzenszwalb, 2004)

4 4 Motivation: good / bad segmentation ADAPTIVE GRAPH-CUT (our)

5 5 Motivation: good / bad segmentation > SLIC (Achanta, 2012)GRAPH-CUT (Felzenszwalb, 2004)ADAPTIVE GRAPH-CUT (our)

6 6 Motivation: good / bad segmentation Achanta, 2012 (SLIC); Kaufhold, 2004: segmentation algorithms aggregate sets of perceptually similar pixels in an image. Felzenszwalb, 2004 (graph-cut): a segmentation algorithm should capture perceptually important groupings or regions, which often reflect global aspects of the image.

7 7 Motivation: segmentation & video compression Segment motion estimation Frame segmentation Encoding True block and sub-block motion vectors

8 8 Aim #1: use the human factor (aka segmentation quality metric)

9 9 Aim #2: automatic parameter tuning

10 10 Road map 1) Pick a segmentation algorithm… 2) … Learn a quality metric including the human factor (application needs) … 3) … And put them together (autotuning).

11 11 Graph: Nodes: Edges: Weights: vivi vjvj w(v i, v j )=0 w(v i, v j )>0 Graph-cut w(v i, v j )>>0

12 12 Internal difference: Graph-cut CmCm

13 13 Difference between components: Graph-cut CmCm CnCn

14 14 Boundary predicate: Graph-cut CkCk CnCn

15 15 Graph-cut C1C1 C2C2 Boundary predicate:

16 16 Graph-cut C1C1 C2C2 Boundary predicate: Observation scale ~ k

17 17 Graph-cut K = 3 K = 100 K = 10,000

18 18 Road map 1) Pick a segmentation algorithm… 2) … Learn a quality metric including the human factor… 3) … And put them together (autotuning).

19 19 (Weighted) symmetric uncertainty 4 bits = 33% 7 bits + 5 bits Entropy based average

20 20 k vs. U w vs. quality 160 x 120 image block

21 21 k vs. U w vs. quality Training 160 x 120 blocks 320x240 rgb images K = [1, …, 10,000] visual inspection & classification

22 22 k vs. U w vs. quality Training 160 x 120 blocks 640x480 rgb images K = [1, …, 10,000] visual inspection & classification

23 23 Learning the metric U w = m log(k) + b

24 24 Road map 1) Pick a segmentation algorithm… 2) … Learn a quality metric including the human factor… 3) … And put them together (autotuning).

25 25 Automatic k selection

26 26 Automatic k selection

27 27 Automatic k selection

28 28 Automatic k selection

29 29 Automatic k selection

30 30 … and adaptivity k = k(x,y)

31 31 Road map

32 32 Results - Quality Adaptive graph-cut (ours) Graph-cut (Felzensswalb, 2004) * SLIC (Achanta, 2012) * * Same number of segments forced for each algorithm

33 33 Results

34 34 Results SLIC Graph-cut Adaptive graph-cut

35 35 Results

36 36 Results SLIC Graph-cut Adaptive graph-cut

37 37 Results: inter-class contrast (the higher the better) Sum of the contrasts among segments weighted by their areas (Chabrier, 2004)

38 38 Results: intra-class uniformity (the lower the better) Sum of the normalized standard deviation for each region (Chabrier, 2004)

39 39 Results: contrast - uniformity ratio (the higher the better)

40 40 Discussion LEARNED segmentation quality metric including the HUMAN FACTOR Iterative method to AUTOMATICALLY and ADAPTIVELY compute the optimal scale parameter

41 41 A more general approach (edge thresholding segmentation in YUV)

42 42 A more general approach (edge thresholding segmentation in YUV) Openboradcast encoding (x264) Lyricallabs encoding (adaptive segmentation) Show

43 43 A more general approach (edge thresholding segmentation in YUV) Openboradcast encoding (x264)Lyricallabs encoding (adaptive segmentation) Show

44 44 Open issues & improvements Resolution dependency (160x120 blocks) Learning: the Berkeley Segmentation Dataset Avoid iterations (see I. Frosio, SPIE EI 2015)

45 45 Questions


Download ppt "Adaptive Segmentation Based on a Learned Quality Metric I. Frosio 1, E. Ratner 2 1 NVIDIA, USA, 2 Lyrical Labs, USA."

Similar presentations


Ads by Google