1. ENSEMBLE SEGMENTATION USING EFFICIENT INTEGER LINEAR PROGRAMMING Ju-Hsin Hsieh Advisor : Sheng-Jyh Wang 2013/07/22 Amir Alush and Jacob Goldberger, “ Ensemble Segmentation Using Efficient Integer Linear Programming ”, IEEE Transactions on PAMI, 2012.

2. Outline ◦Introduction ◦Method ◦Experiment result ◦Conclusion ◦Reference 2

3. Outline ◦Introduction What is segmentation? Challenge Main idea ◦Method ◦Experiment result ◦Conclusion ◦Reference 3

4. What is segmentation? ◦Partitioning of an image into several constituent components. ◦Assign each pixel in the image to one of the image components. 4

5. Outline ◦Introduction What is segmentation? Challenge Main idea ◦Method ◦Experiment result ◦Conclusion ◦Reference 5

6. Challenge ◦Segmentation is not a well-defined task. 6

7. Challenge ◦Segmentations have different numbers of segments and are inconsistent. ◦How to estimate the quality of each segmentation algorithm in an unsupervised manner? 34 segments 77 segments 7

8. Outline ◦Introduction What is segmentation? Challenge Main idea ◦Method ◦Experiment result ◦Conclusion ◦Reference 8

9. Main idea ◦Combine segmentations of the same image obtained by different algorithms. ◦Average of all the segmentations. ◦The quality of segmentation is based on the consistency of the segmentation compared to the other algorithms. 9

10. Main idea 0.93 0.69 0.65 0.93 0.74 0.70 Average segmentation Input image 10

11. Outline ◦Introduction ◦Method Probabilistic framework - Definition - EM algorithm Integer Linear Programming Processing Procedure Additional information ◦Experiment result ◦Conclusion ◦Reference 11

12. Probabilistic framework 12

13. Probabilistic framework 13

14. Outline ◦Introduction ◦Method Probabilistic framework - Definition - EM algorithm Integer Linear Programming Processing Procedure Additional information ◦Experiment result ◦Conclusion ◦Reference 14

15. Probabilistic framework 15

16. Probabilistic framework 16

17. Probabilistic framework 17

18. Outline ◦Introduction ◦Method Probabilistic framework - Definition - EM algorithm Integer Linear Programming Processing Procedure Additional information ◦Experiment result ◦Conclusion ◦Reference 18

19. Integer Linear Programming 19

20. Integer Linear Programming Transitive relation If x ij = x jk = 1 then x ik = 1 The complexity of ILP is high. 20

21. Outline ◦Introduction ◦Method Probabilistic framework - Definition - EM algorithm Integer Linear Programming Processing Procedure Additional information ◦Experiment result ◦Conclusion ◦Reference 21

22. Processing Procedure G = ( V, E ) with { w ij } 1. Divided into “positively connected components” Negative weight Positive weight 2. Transform to “Single Edge Partition Tree” 3. Divided into subgraphs 22

23. Processing Procedure G = ( V, E ) with { w ij } 1. Divided into “positively connected components” 2. Transform to “Single Edge Partition Tree” 3. Divided into subgraphs 23

24. Processing Procedure c (V 1,E 1 ) c (V 2,E 2 ) Crossing edge E 12 G( V, E ) Negative edge G( V, E ) 24

25. Processing Procedure 1. Divided into “positively connected components” ◦Approach 25

26. Processing Procedure G = ( V, E ) with { w ij } 1. Divided into “positively connected components” 2. Transform to “Single Edge Partition Tree” 3. Divided into subgraphs 26

27. Processing Procedure 2. Transform to “Single Edge Partition Tree” ◦Approach Case 1 Cycle-free graph(tree) V1V1 V2V2 V3V3 V4V4 V V1V1 V2V2 V3V3 V4V4 V5V5 27

28. Processing Procedure 2. Transform to “Single Edge Partition Tree” ◦Approach Case 2 V1V1 V2V2 V3V3 V4V4 V V1V1 V2V2 V3V3 V4V4 28

29. Processing Procedure 2. Transform to “Single Edge Partition Tree” ◦Approach Case 3 V1V1 V2V2 V3V3 V4V4 V V1V1 V3V3 V4V4 29

30. Processing Procedure G = ( V, E ) with { w ij } 1. Divided into “positively connected components” 2. Transform to “Single Edge Partition Tree” 3. Divided into subgraphs 30

31. Processing Procedure 3. Divided into subgraphs V1V1 V2V2 V3V3 V4V4 V5V5 V1V1 V2V2 V3V3 V4V4 V5V5 31

32. Outline ◦Introduction ◦Method Probabilistic framework - Definition - EM algorithm Integer Linear Programming Processing Procedure Additional information ◦Experiment result ◦Conclusion ◦Reference 32

33. Additional information ◦Image spatial consistency  Neighboring pixels are more likely to be in the same cluster than pixels that are far apart. ◦Approach  Use mean-shift algorithm to oversegment the image into small, homogeneous regions, known as superpixels.  Merging the MS superpixels, based on consensus among the experts. 33

34. Averaging Multiple Unreliable Segmentations ( AMUS ) AMUS 0.93 0.69 0.65 0.93 0.74 0.70Averaging Segmentation 34

35. Averaging Multiple Unreliable Segmentations ( AMUS ) G = ( V, E ) with { w ij } Divided into “positively connected components” Transform to “Single Edge Partition Tree” Divided into subgraphs Use MS to get superpixels Apply ILP to each subgraphs 35

36. Outline ◦Introduction ◦Method ◦Experiment result AMUS algorithm Compare with other algorithms ◦Conclusion ◦Reference 36

37. AMUS algorithm 0.73 0.620.74 0.95 0.87 0.89 Result Averaging segmentation 37

38. Outline ◦Introduction ◦Method ◦Experiment result AMUS algorithm Compare with other algorithms ◦Conclusion ◦Reference 38

39. Compare with other algorithms Image AMUS CTM TBES MNC UCM PRI(probabilistic Rand index) VOI(Variation of information ) GCE(Global Consistency Error) Boundary-based F-measure 39

40. Outline ◦Introduction ◦Method ◦Experiment result AMUS algorithm Compare with other algorithms ◦Conclusion ◦Reference 40

41. Conclusion ◦Segmentation is not a well-defined task. ◦This paper present a method for combining several segmentations of an image into a single one ( the averaging segmentation ) in order to achieve a more reliable and accurate segmentation result. ◦This paper also reports the reliability of each segmentation. 41

42. Outline ◦Introduction ◦Method ◦Experiment result AMUS algorithm Compare with other algorithms ◦Conclusion ◦Reference 42

43. Reference ◦Amir Alush and Jacob Goldberger, “ Ensemble Segmentation Using Efficient Integer Linear Programming ”, IEEE Transactions on PAMI, 2012. 43