Presentation is loading. Please wait.

Presentation is loading. Please wait.

Image Segmentation by Clustering Methods: Cluster Validity Student: Sean Hsien Supervisor: Dr. Sid Ray.

Similar presentations


Presentation on theme: "Image Segmentation by Clustering Methods: Cluster Validity Student: Sean Hsien Supervisor: Dr. Sid Ray."— Presentation transcript:

1 Image Segmentation by Clustering Methods: Cluster Validity Student: Sean Hsien Supervisor: Dr. Sid Ray

2 Overview  Aims of the project  Conventional cluster evaluation  Turi and Ray’s modified criterion  MML and its application in image clustering  2 new evaluation criteria  Test images used  Data acquisition and results  Conclusion  Future work

3 Aims  To test the suitability of Turi and Ray’s modified criterion on greyscale images  Test effectiveness of MML for cluster evaluation in image clustering  To compare different methodologies and find a general and effective way of cluster evaluation

4 Conventional cluster evaluation  Many conventional methods measure cluster compactness and separability — intra/inter cluster distances Inter Intra

5  Based on the basic validity criterion of intra/inter ratio  Modifies the basic criterion with a penalty function to penalize low numbers of clusters Turi and Ray’s Modified Criterion

6 Modified Criterion Formula

7 Minimum Message Length  Principle of Occam’s Razor: don’t make things unnecessarily complex  2-part message length to balance goodness-of-fit and model complexity

8 Application of MML in Image Clustering  Used Segment Map and Complementary Image to apply MML for cluster evaluation: MsgLen(Segment Map)  MsgLen(Model) MsgLen(Complementary Image)  MsgLen(Data|Model)

9 Segmented (3 clusters) Complementary Image Original

10 Compression and MML  Compression with order 1 Hidden Markov Model was used – increase information content, spatial information  Higher order could not be used – large alphabet size of images (256 for greyscale images)

11 2 New Evaluation Criteria  Based on the basic criterion  Desirable to have a penalty function that adapts to the structure of data  Use entropy to indicate cluster tendency where H is the entropy of the input image

12 2 New Evaluation Criteria  Based on the Liu and Yang’s criterion  Their criterion over emphasized goodness-of-fit 2

13 Test Images Used No standard cluster evaluation criteria, therefore:  Use visual assessment of natural image clustering  Generate synthetic images (with/out noise) for qualitative analysis

14 Synthetic Images  Generated noiseless synthetic images with 2, 3, 5, 8, and 15 segments (equally spaced in the range 20 – 235)  Synthetic images with Gaussian noise generated using pgmgauss and noiseless synthetic images as input  Used standard deviation of 2

15 Synthetic Images

16 Natural Images

17 Data acquisition  Used K-means clustering algorithm  Criteria implemented: Basic and its variations, Davies-Bouldin, modified Liu and Yang’s, MML  All criteria performed well with synthetic images – noiseless and noisy

18 Results – Conventional basic( 2): * basic( 3): basic( 4): basic( 5): basic( 6): basic( 7): basic( 8): basic( 9): basic(10): db( 2): db( 3): db( 4): db( 5): db( 6): db( 7): db( 8): db( 9): * db(10): c=25 turiray( 2): turiray( 3): turiray( 4): turiray( 5): turiray( 6): turiray( 7): * turiray( 8): turiray( 9): turiray(10): mly( 2): 2.982e+03 mly( 3): 3.422e+03 mly( 4): 3.497e+03 mly( 5): 4.193e+03 mly( 6): 3.493e+03 mly( 7): 2.684e+03 mly( 8): 1.850e+03 mly( 9): 1.749e+03* mly(10): 1.821e+03 c=3 turiray( 2): turiray( 3): turiray( 4): * turiray( 5): turiray( 6): turiray( 7): turiray( 8): turiray( 9): turiray(10): newb( 2): * newb( 3): newb( 4): newb( 5): newb( 6): newb( 7): newb( 8): newb( 9): newb(10): noisy-gauss- horiz.gif

19 Results – MML #clusters( 2) msglen(seg): msglen(comp): total: #clusters( 3) msglen(seg): msglen(comp): total: #clusters( 4) msglen(seg): msglen(comp): total: #clusters( 5) msglen(seg): msglen(comp): total: #clusters( 6) msglen(seg): msglen(comp): total: #clusters( 7) msglen(seg): msglen(comp): total: #clusters( 8) msglen(seg): msglen(comp): total: * #clusters( 9) msglen(seg): msglen(comp): total: #clusters(10) msglen(seg): msglen(comp): total: #clusters(11) msglen(seg): msglen(comp): total: #clusters(12) msglen(seg): msglen(comp): total: #clusters(13) msglen(seg): msglen(comp): total: #clusters(14) msglen(seg): msglen(comp): total: #clusters(15) msglen(seg): msglen(comp): total: #clusters(16) msglen(seg): msglen(comp): total: #clusters(17) msglen(seg): msglen(comp): total: #clusters(18) msglen(seg): msglen(comp): total: #clusters(19) msglen(seg): msglen(comp): total: #clusters(20) msglen(seg): msglen(comp): total: noisy-gauss- horiz.gif

20 Results – MML (compression) #clusters( 2) msglen(seg): msglen(comp): total: #clusters( 3) msglen(seg): msglen(comp): total: #clusters( 4) msglen(seg): msglen(comp): total: #clusters( 5) msglen(seg): msglen(comp): total: #clusters( 6) msglen(seg): msglen(comp): total: #clusters( 7) msglen(seg): msglen(comp): total: * #clusters( 8) msglen(seg): msglen(comp): total: #clusters( 9) msglen(seg): msglen(comp): total: #clusters(10) msglen(seg): msglen(comp): total: #clusters(11) msglen(seg): msglen(comp): total: #clusters(12) msglen(seg): msglen(comp): total: #clusters(13) msglen(seg): msglen(comp): total: #clusters(14) msglen(seg): msglen(comp): total: #clusters(15) msglen(seg): msglen(comp): total: #clusters(16) msglen(seg): msglen(comp): total: #clusters(17) msglen(seg): msglen(comp): total: #clusters(18) msglen(seg): msglen(comp): total: #clusters(19) msglen(seg): msglen(comp): total: #clusters(20) msglen(seg): msglen(comp): total: noisy-gauss- horiz.gif

21 Results – Conventional basic( 2): * basic( 3): basic( 4): basic( 5): basic( 6): basic( 7): basic( 8): basic( 9): basic(10): db( 2): * db( 3): db( 4): db( 5): db( 6): db( 7): db( 8): db( 9): db(10): c=25 turiray( 2): turiray( 3): turiray( 4): turiray( 5): turiray( 6): * turiray( 7): turiray( 8): turiray( 9): turiray(10): mly( 2): 2.901e+03* mly( 3): 3.556e+03 mly( 4): 3.794e+03 mly( 5): 4.109e+03 mly( 6): 4.343e+03 mly( 7): 4.347e+03 mly( 8): 4.395e+03 mly( 9): 4.407e+03 mly(10): 4.482e+03 c=3 turiray( 2): * turiray( 3): turiray( 4): turiray( 5): turiray( 6): turiray( 7): turiray( 8): turiray( 9): turiray(10): newb( 2): * newb( 3): newb( 4): newb( 5): newb( 6): newb( 7): newb( 8): newb( 9): newb(10): pellets.gif

22 Results – MML #clusters( 2) msglen(seg): msglen(comp): total: < #clusters( 3) msglen(seg): msglen(comp): total: #clusters( 4) msglen(seg): msglen(comp): total: #clusters( 5) msglen(seg): msglen(comp): total: #clusters( 6) msglen(seg): msglen(comp): total: #clusters( 7) msglen(seg): msglen(comp): total: #clusters( 8) msglen(seg): msglen(comp): total: #clusters( 9) msglen(seg): msglen(comp): total: #clusters(10) msglen(seg): msglen(comp): total: #clusters(11) msglen(seg): msglen(comp): total: #clusters(12) msglen(seg): msglen(comp): total: #clusters(13) msglen(seg): msglen(comp): total: #clusters(14) msglen(seg): msglen(comp): total: #clusters(15) msglen(seg): msglen(comp): total: #clusters(16) msglen(seg): msglen(comp): total: #clusters(17) msglen(seg): msglen(comp): total: #clusters(18) msglen(seg): msglen(comp): total: #clusters(19) msglen(seg): msglen(comp): total: #clusters(20) msglen(seg): msglen(comp): total: * pellets.gif

23 Results – MML (compression) #clusters( 2) msglen(seg): msglen(comp): total: * #clusters( 3) msglen(seg): msglen(comp): total: #clusters( 4) msglen(seg): msglen(comp): total: #clusters( 5) msglen(seg): msglen(comp): total: #clusters( 6) msglen(seg): msglen(comp): total: #clusters( 7) msglen(seg): msglen(comp): total: #clusters( 8) msglen(seg): msglen(comp): total: #clusters( 9) msglen(seg): msglen(comp): total: #clusters(10) msglen(seg): msglen(comp): total: #clusters(11) msglen(seg): msglen(comp): total: #clusters(12) msglen(seg): msglen(comp): total: #clusters(13) msglen(seg): msglen(comp): total: #clusters(14) msglen(seg): msglen(comp): total: #clusters(15) msglen(seg): msglen(comp): total: #clusters(16) msglen(seg): msglen(comp): total: #clusters(17) msglen(seg): msglen(comp): total: #clusters(18) msglen(seg): msglen(comp): total: #clusters(19) msglen(seg): msglen(comp): total: #clusters(20) msglen(seg): msglen(comp): total: pellets.gif

24 Results – Conventional basic( 2): basic( 3): basic( 4): basic( 5): basic( 6): * basic( 7): basic( 8): basic( 9): basic(10): db( 2): db( 3): db( 4): db( 5): db( 6): * db( 7): db( 8): db( 9): db(10): c=25 turiray( 2): turiray( 3): turiray( 4): turiray( 5): turiray( 6): * turiray( 7): turiray( 8): turiray( 9): turiray(10): mly( 2): 1.481e+04 mly( 3): 1.352e+04 mly( 4): 1.404e+04 mly( 5): 1.412e+04 mly( 6): 1.314e+04* mly( 7): 1.419e+04 mly( 8): 1.381e+04 mly( 9): 1.396e+04 mly(10): 1.496e+04 c=3 turiray( 2): turiray( 3): turiray( 4): turiray( 5): turiray( 6): * turiray( 7): turiray( 8): turiray( 9): turiray(10): newb( 2): * newb( 3): newb( 4): newb( 5): newb( 6): newb( 7): newb( 8): newb( 9): newb(10): mug.gif

25 Results – MML #clusters( 2) msglen(seg): msglen(comp): total: #clusters( 3) msglen(seg): msglen(comp): total: * #clusters( 4) msglen(seg): msglen(comp): total: #clusters( 5) msglen(seg): msglen(comp): total: #clusters( 6) msglen(seg): msglen(comp): total: #clusters( 7) msglen(seg): msglen(comp): total: #clusters( 8) msglen(seg): msglen(comp): total: #clusters( 9) msglen(seg): msglen(comp): total: #clusters(10) msglen(seg): msglen(comp): total: #clusters(11) msglen(seg): msglen(comp): total: #clusters(12) msglen(seg): msglen(comp): total: #clusters(13) msglen(seg): msglen(comp): total: #clusters(14) msglen(seg): msglen(comp): total: #clusters(15) msglen(seg): msglen(comp): total: #clusters(16) msglen(seg): msglen(comp): total: #clusters(17) msglen(seg): msglen(comp): total: #clusters(18) msglen(seg): msglen(comp): total: #clusters(19) msglen(seg): msglen(comp): total: #clusters(20) msglen(seg): msglen(comp): total: mug.gif

26 Results – MML (compression) #clusters( 2) msglen(seg): msglen(comp): total: * #clusters( 3) msglen(seg): msglen(comp): total: #clusters( 4) msglen(seg): msglen(comp): total: #clusters( 5) msglen(seg): msglen(comp): total: #clusters( 6) msglen(seg): msglen(comp): total: #clusters( 7) msglen(seg): msglen(comp): total: #clusters( 8) msglen(seg): msglen(comp): total: #clusters( 9) msglen(seg): msglen(comp): total: #clusters(10) msglen(seg): msglen(comp): total: #clusters(11) msglen(seg): msglen(comp): total: #clusters(12) msglen(seg): msglen(comp): total: #clusters(13) msglen(seg): msglen(comp): total: #clusters(14) msglen(seg): msglen(comp): total: #clusters(15) msglen(seg): msglen(comp): total: #clusters(16) msglen(seg): msglen(comp): total: #clusters(17) msglen(seg): msglen(comp): total: #clusters(18) msglen(seg): msglen(comp): total: #clusters(19) msglen(seg): msglen(comp): total: #clusters(20) msglen(seg): msglen(comp): total: mug.gif

27 Results – Conventional basic( 2): * basic( 3): basic( 4): basic( 5): basic( 6): basic( 7): basic( 8): basic( 9): basic(10): db( 2): * db( 3): db( 4): db( 5): db( 6): db( 7): db( 8): db( 9): db(10): c=25 turiray( 2): turiray( 3): turiray( 4): turiray( 5): turiray( 6): turiray( 7): turiray( 8): turiray( 9): * turiray(10): mly( 2): 6.829e+03* mly( 3): 8.138e+03 mly( 4): 9.085e+03 mly( 5): 8.234e+03 mly( 6): 8.682e+03 mly( 7): 8.373e+03 mly( 8): 8.747e+03 mly( 9): 8.590e+03 mly(10): 8.723e+03 c=3 turiray( 2): * turiray( 3): turiray( 4): turiray( 5): turiray( 6): turiray( 7): turiray( 8): turiray( 9): turiray(10): newb( 2): * newb( 3): newb( 4): newb( 5): newb( 6): newb( 7): newb( 8): newb( 9): newb(10): parts.gif

28 Results – MML #clusters( 2) msglen(seg): msglen(comp): total: #clusters( 3) msglen(seg): msglen(comp): total: #clusters( 4) msglen(seg): msglen(comp): total: #clusters( 5) msglen(seg): msglen(comp): total: #clusters( 6) msglen(seg): msglen(comp): total: #clusters( 7) msglen(seg): msglen(comp): total: #clusters( 8) msglen(seg): msglen(comp): total: #clusters( 9) msglen(seg): msglen(comp): total: #clusters(10) msglen(seg): msglen(comp): total: #clusters(11) msglen(seg): msglen(comp): total: #clusters(12) msglen(seg): msglen(comp): total: #clusters(13) msglen(seg): msglen(comp): total: #clusters(14) msglen(seg): msglen(comp): total: #clusters(15) msglen(seg): msglen(comp): total: #clusters(16) msglen(seg): msglen(comp): total: * #clusters(17) msglen(seg): msglen(comp): total: #clusters(18) msglen(seg): msglen(comp): total: #clusters(19) msglen(seg): msglen(comp): total: #clusters(20) msglen(seg): msglen(comp): total: parts.gif

29 Results – MML (compression) #clusters( 2) msglen(seg): msglen(comp): total: * #clusters( 3) msglen(seg): msglen(comp): total: #clusters( 4) msglen(seg): msglen(comp): total: #clusters( 5) msglen(seg): msglen(comp): total: #clusters( 6) msglen(seg): msglen(comp): total: #clusters( 7) msglen(seg): msglen(comp): total: #clusters( 8) msglen(seg): msglen(comp): total: #clusters( 9) msglen(seg): msglen(comp): total: #clusters(10) msglen(seg): msglen(comp): total: #clusters(11) msglen(seg): msglen(comp): total: #clusters(12) msglen(seg): msglen(comp): total: #clusters(13) msglen(seg): msglen(comp): total: #clusters(14) msglen(seg): msglen(comp): total: #clusters(15) msglen(seg): msglen(comp): total: #clusters(16) msglen(seg): msglen(comp): total: #clusters(17) msglen(seg): msglen(comp): total: #clusters(18) msglen(seg): msglen(comp): total: #clusters(19) msglen(seg): msglen(comp): total: #clusters(20) msglen(seg): msglen(comp): total: parts.gif

30 Results – Conventional basic( 2): basic( 3): * basic( 4): basic( 5): basic( 6): basic( 7): basic( 8): basic( 9): basic(10): db( 2): db( 3): * db( 4): db( 5): db( 6): db( 7): db( 8): db( 9): db(10): c=25 turiray( 2): turiray( 3): turiray( 4): turiray( 5): turiray( 6): * turiray( 7): turiray( 8): turiray( 9): turiray(10): mly( 2): 1.244e+04 mly( 3): 2.853e+03* mly( 4): 3.162e+03 mly( 5): 3.780e+03 mly( 6): 3.758e+03 mly( 7): 4.277e+03 mly( 8): 4.304e+03 mly( 9): 4.392e+03 mly(10): 4.925e+03 c=3 turiray( 2): turiray( 3): turiray( 4): * turiray( 5): turiray( 6): turiray( 7): turiray( 8): turiray( 9): turiray(10): newb( 2): newb( 3): * newb( 4): newb( 5): newb( 6): newb( 7): newb( 8): newb( 9): newb(10): catscan.gif

31 Results – MML #clusters( 2) msglen(seg): msglen(comp): total: * #clusters( 3) msglen(seg): msglen(comp): total: #clusters( 4) msglen(seg): msglen(comp): total: #clusters( 5) msglen(seg): msglen(comp): total: #clusters( 6) msglen(seg): msglen(comp): total: #clusters( 7) msglen(seg): msglen(comp): total: #clusters( 8) msglen(seg): msglen(comp): total: #clusters( 9) msglen(seg): msglen(comp): total: #clusters(10) msglen(seg): msglen(comp): total: #clusters(11) msglen(seg): msglen(comp): total: #clusters(12) msglen(seg): msglen(comp): total: #clusters(13) msglen(seg): msglen(comp): total: #clusters(14) msglen(seg): msglen(comp): total: #clusters(15) msglen(seg): msglen(comp): total: #clusters(16) msglen(seg): msglen(comp): total: #clusters(17) msglen(seg): msglen(comp): total: #clusters(18) msglen(seg): msglen(comp): total: #clusters(19) msglen(seg): msglen(comp): total: #clusters(20) msglen(seg): msglen(comp): total: catscan.gif

32 Results – MML (compression) #clusters( 2) msglen(seg): msglen(comp): total: * #clusters( 3) msglen(seg): msglen(comp): total: #clusters( 4) msglen(seg): msglen(comp): total: #clusters( 5) msglen(seg): msglen(comp): total: #clusters( 6) msglen(seg): msglen(comp): total: #clusters( 7) msglen(seg): msglen(comp): total: #clusters( 8) msglen(seg): msglen(comp): total: #clusters( 9) msglen(seg): msglen(comp): total: #clusters(10) msglen(seg): msglen(comp): total: #clusters(11) msglen(seg): msglen(comp): total: #clusters(12) msglen(seg): msglen(comp): total: #clusters(13) msglen(seg): msglen(comp): total: #clusters(14) msglen(seg): msglen(comp): total: #clusters(15) msglen(seg): msglen(comp): total: #clusters(16) msglen(seg): msglen(comp): total: #clusters(17) msglen(seg): msglen(comp): total: #clusters(18) msglen(seg): msglen(comp): total: #clusters(19) msglen(seg): msglen(comp): total: #clusters(20) msglen(seg): msglen(comp): total: catscan.gif

33 Results – Conventional basic( 2): basic( 3): basic( 4): * basic( 5): basic( 6): basic( 7): basic( 8): basic( 9): basic(10): db( 2): db( 3): db( 4): db( 5): * db( 6): db( 7): db( 8): db( 9): db(10): c=25 turiray( 2): turiray( 3): turiray( 4): turiray( 5): turiray( 6): turiray( 7): * turiray( 8): turiray( 9): turiray(10): mly( 2): 1.141e+04 mly( 3): 1.215e+04 mly( 4): 1.105e+04 mly( 5): 1.066e+04* mly( 6): 1.138e+04 mly( 7): 1.099e+04 mly( 8): 1.114e+04 mly( 9): 1.154e+04 mly(10): 1.140e+04 c=3 turiray( 2): turiray( 3): turiray( 4): turiray( 5): * turiray( 6): turiray( 7): turiray( 8): turiray( 9): turiray(10): newb( 2): * newb( 3): newb( 4): newb( 5): newb( 6): newb( 7): newb( 8): newb( 9): newb(10): lenna_256.gif

34 Results – MML #clusters( 2) msglen(seg): msglen(comp): total: #clusters( 3) msglen(seg): msglen(comp): total: #clusters( 4) msglen(seg): msglen(comp): total: * #clusters( 5) msglen(seg): msglen(comp): total: #clusters( 6) msglen(seg): msglen(comp): total: #clusters( 7) msglen(seg): msglen(comp): total: #clusters( 8) msglen(seg): msglen(comp): total: #clusters( 9) msglen(seg): msglen(comp): total: #clusters(10) msglen(seg): msglen(comp): total: #clusters(11) msglen(seg): msglen(comp): total: #clusters(12) msglen(seg): msglen(comp): total: #clusters(13) msglen(seg): msglen(comp): total: #clusters(14) msglen(seg): msglen(comp): total: #clusters(15) msglen(seg): msglen(comp): total: #clusters(16) msglen(seg): msglen(comp): total: #clusters(17) msglen(seg): msglen(comp): total: #clusters(18) msglen(seg): msglen(comp): total: #clusters(19) msglen(seg): msglen(comp): total: #clusters(20) msglen(seg): msglen(comp): total: lenna_256.gif

35 Results – MML (compression) #clusters( 2) msglen(seg): msglen(comp): total: * #clusters( 3) msglen(seg): msglen(comp): total: #clusters( 4) msglen(seg): msglen(comp): total: #clusters( 5) msglen(seg): msglen(comp): total: #clusters( 6) msglen(seg): msglen(comp): total: #clusters( 7) msglen(seg): msglen(comp): total: #clusters( 8) msglen(seg): msglen(comp): total: #clusters( 9) msglen(seg): msglen(comp): total: #clusters(10) msglen(seg): msglen(comp): total: #clusters(11) msglen(seg): msglen(comp): total: #clusters(12) msglen(seg): msglen(comp): total: #clusters(13) msglen(seg): msglen(comp): total: #clusters(14) msglen(seg): msglen(comp): total: #clusters(15) msglen(seg): msglen(comp): total: #clusters(16) msglen(seg): msglen(comp): total: #clusters(17) msglen(seg): msglen(comp): total: #clusters(18) msglen(seg): msglen(comp): total: #clusters(19) msglen(seg): msglen(comp): total: #clusters(20) msglen(seg): msglen(comp): total: lenna_256.gif

36  Assumes normally distributed clusters (use of Euclidean distance imply spherical clusters)  Intra approaches 0 as number of segments approach number of grey levels in image  High correlation between intra and inter cluster distances Conclusion – Conventional Methods

37  The value of c does matter with the evaluation of clusters from greyscale images  For greyscale images, optimum value of c was found to be 3 Conclusion – Turi and Ray’s Modified Criterion

38 Conclusion – New Evaluation Methods  New basic: Too much bias towards the low number of clusters  Modified Liu and Yang: Found to give same results as Davies-Bouldin index with only intra- cluster information – shows the high correlation between intra and inter

39  Makes no assumptions  MML provides a general way of qualitative cluster assessment  MsgLen(segment map) may decrease with increasing K  Compression seems to bias smaller number of clusters  Markovian compression impractical – large alphabet size of images (preprocessing) Conclusion – MML

40 Future Work  Test MML and new approaches with colour images (various colour spaces)  Use other compression or noise removal techniques to improve MML analysis  Use of different clustering algorithms to form other types of cluster distributions  Explore further adaptive penalty functions

41 Questions?


Download ppt "Image Segmentation by Clustering Methods: Cluster Validity Student: Sean Hsien Supervisor: Dr. Sid Ray."

Similar presentations


Ads by Google