1. City University of Hong Kong 18 th Intl. Conf. Pattern Recognition Self-Validated and Spatially Coherent Clustering with NS-MRF and Graph Cuts Wei Feng and Zhi-Qiang Liu Group of Media Computing School of Creative Media City University of Hong Kong

2. 18 th Intl. Conf. Pattern Recognition Outline Motivation Related Work Proposed Method Results Discussion

3. City University of Hong Kong 18 th Intl. Conf. Pattern Recognition Clustering in Low Level Vision Common problem: segmentation, stereo etc. Two parts should be considered:  Accuracy (i.e., likelihood)  Spatial coherence (i.e., cost) Bayesian framework: to minimize the Gibbs energy (equivalent form of MAP)

4. City University of Hong Kong 18 th Intl. Conf. Pattern Recognition Motivation Computational complexity remains a major weakness of the MRF/MAP scheme How to determine the number of clusters (i.e., self-validation)

5. City University of Hong Kong 18 th Intl. Conf. Pattern Recognition Related Work Interactive segmentation [Boykov, ICCV’01] Lazy snapping [Li, SIGGRAPH’03] Mean shift [Comaniciu and Meer, 02] TS-MRF [D’Elia, 03] Graph based segmentation [Felzenszwalb, 04] Spatial coherence clustering [Zabih, 04] …

6. City University of Hong Kong 18 th Intl. Conf. Pattern Recognition Solving Binary MRF with Graph Mincut For a binary MRF, the optimal labeling can be achieved by graph mincut Likelihood energy Coherence energy

7. City University of Hong Kong 18 th Intl. Conf. Pattern Recognition Feature Samples Representation Non-parametric representation:

8. City University of Hong Kong 18 th Intl. Conf. Pattern Recognition Energy Assignment Based on the two components C 0 and C 1 and their corresponding subcomponents M 0 k and M 1 k, we can define likelihood energy and coherence energy in a nonparametric form. Modified Potts Model

9. City University of Hong Kong 18 th Intl. Conf. Pattern Recognition NS-MRF Net-Structured MRF  A powerful tool for labeling problems in low level vision  An efficient energy minimization scheme by graph cuts  Converting the K-class clustering into a sequence of K−1 much simpler binary clustering

10. City University of Hong Kong 18 th Intl. Conf. Pattern Recognition Energy Assignment for NS-MRF Cluster Remaining Energy: Cluster Merging Energy: Cluster Splitting Energy: Cluster Coherence Energy:

11. City University of Hong Kong 18 th Intl. Conf. Pattern Recognition Optimal Cluster Evolution

12. City University of Hong Kong 18 th Intl. Conf. Pattern Recognition Cluster Evolution

13. City University of Hong Kong 18 th Intl. Conf. Pattern Recognition Image Segmentation via NS-MRF The preservation of soft edges: [1] P. F. Felzenszwalb and D. P. Huttenlocher. “Efficient graph based image segmentation”, IJCV 2004. [2] D. Comaniciu and P. Meer. “Mean shift: A robust approach towards feature space analysis”, PAMI 2002. [1] [2]

14. City University of Hong Kong 18 th Intl. Conf. Pattern Recognition Image Segmentation via NS-MRF The robustness to noise: [1] C. D’Elia et al. “A tree-structured markov random field model for bayesian image segmentation”, IEEE Trans. Image Processing 2003. [2] P. F. Felzenszwalb and D. P. Huttenlocher. “Efficient graph based image segmentation”, IJCV 2004. [3] D. Comaniciu and P. Meer. “Mean shift: A robust approach towards feature space analysis”, PAMI 2002. [2] [3] [1]

15. City University of Hong Kong 18 th Intl. Conf. Pattern Recognition More Results

16. City University of Hong Kong 18 th Intl. Conf. Pattern Recognition More Results

17. City University of Hong Kong 18 th Intl. Conf. Pattern Recognition More Results

18. City University of Hong Kong 18 th Intl. Conf. Pattern Recognition More Results

19. City University of Hong Kong 18 th Intl. Conf. Pattern Recognition More Results

20. City University of Hong Kong 18 th Intl. Conf. Pattern Recognition Discussion NS-MRF is an efficient clustering method which is self-validated and guarantees stepwise global optimum. It is ready to apply to a wide range of clustering problems in low-level vision. Future work:  clustering bias  multi-resolution graph construction scheme for graph cuts based image modeling

21. City University of Hong Kong 18 th Intl. Conf. Pattern Recognition Thanks!