1. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 1 Ivan Kyrgyzov, Henri Maître Télécom Paris 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Images

2. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 2 Plan of presentation 1.Problem statement 2.Unsupervised clustering algorithms 3.The optimal number of clusters (MDL criterion) 4.Usupervised clustering combination 5.General schema of image clustering 6.Image data 7.Results

3. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 3 1. Problem statement Satellite Image Analysis as a Pattern Recognition Problem Questions: Problem: different algorithms give different clusterings different views of data representations some clusters are common, some clusters are unique Method: unsupervised clustering algorithms 1) Do not search classes manually. 2) Descriptors form clusters. Solution: combine results of clustering algorithms and find a consensus Which patterns? How many pattern prototypes? How to discriminate them?

4. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 4 2. Unsupervised clustering algorithms EM-algorithm for GMM estimation (Autoclass, Cheeseman, 1996) Kernel K-means algorithm (Taylor & Cristianini 2004) Spectral K-means algorithm (A.Ng, 2002 ) Ward hierarchical algorithm (Ward, 1963) Combining different clusterings using a co-association matrix

5. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 5 3. The optimal number of clusters (MDL) Minimum Description Length (Rissanen, 1978) (1) (2) P(X|Θ) of Gaussians and a “hard clustering” (1) is: = number of samples in i th cluster, = determinant of covariance matrix calculated both for spectral and kernel clustering algorithms

6. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 6 4. Unsupervised clustering combination Co-association matrix Fred & Jain (PAMI 2005) Final combination =, such that: (3) Combination algorithm: single-link based clustering (1974, Hubert) to minimize (3)

7. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 7 5. General schema of image clustering SPOT5 images Samples Image data Models Model 1 Model 2 Selected Models SModel 1 SModel 2 Clustering 1 Clustering 2 Sample extraction Feature extraction Unsupervised feature selection Unsupervised clustering Unsupervised combination Consensus clustering

8. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 8 6. Image data SPOT5, 5 m/pixel (64x64 pixels) 529 samples per image Sample extraction ©CNES Original images Database of samples Sample features HaralickGeometricalQMFGabor Sample 1... Sample 2... Feature extraction and selection

9. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 9 6.1. Image data SPOT5, 5 m/pixel Barcelona Los Angeles Istanbul Madrid Image examples of world cities

10. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 10 7. Results. Clustering by Autoclass Image examples of world citiesEstimated number of clusters: 14 Barcelona Los Angeles Istanbul Madrid

11. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 11 7.1. Clustering by Kernel K-means Optimal number of clusters: 7

12. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 12 7.2. Clustering by Kernel K-means Image examples of world citiesOptimal number of clusters: 7 Barcelona Los Angeles Istanbul Madrid

13. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 13 7.3. Clustering by Spectral K-means Optimal number of clusters: 9

14. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 14 7.4. Clustering by Spectral K-means Image examples of world citiesOptimal number of clusters: 9 Barcelona Los Angeles Istanbul Madrid

15. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 15 7.5. Clustering by Ward algorithm Optimal number of clusters: 4

16. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 16 7.6. Clustering by Ward algorithm Image examples of world citiesOptimal number of clusters: 4 Barcelona Los Angeles Istanbul Madrid

17. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 17 7.7. Unsupervised combination Optimal number of clusters: 12

18. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 18 7.8. Unsupervised combination Image examples of world citiesOptimal number of clusters: 12 Barcelona Los Angeles Istanbul Madrid

19. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 19 Conclusions  consensus solution which reduces redundant information in clusterings  relevant unsupervised approach  interpretable clusters and their relations  generalizes information and helps to understand an image context  Combination of different optimal clusterings:

20. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 20 Perspectives -Unsupervised combination can be applied to data with different clustering algorithms and metrcis, groups of features, classifications, segmentations, maps, labellings, temporal images, etc., … - Using clusters for semi-supervised tasks Construction of a satellite image semantic Propose to the user images from different clusters which will be associated with semantic terms by human interpretation.

21. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings 21 Thank you for your attention! Questions ?