Map Asia 2003 M. İzzet SAĞLAM WELCOMEWELCOME CLASSIFICATION OF SATELLITE IMAGES BY USING SELF ORGANIZING MAP AND LINEAR SUPPORT VECTOR MACHINE DECISION TREE (SOM-LSVMDT) M. IZZET SAGLAM ITU ADVANCED TECHNOLOGIES IN ENGINEERING KUALA LUMPUR 2003

Map Asia 2003 M. İzzet SAĞLAM CONTENTSCONTENTS Introduction The Classification Problem  Problem Description  Complexity Description Linear Support Vector Machine Decision Tree (LSVMDT) The Proposed Algorithm (SOM-LSVMDT) Experimental Results and Conclusions

Map Asia 2003 M. İzzet SAĞLAM INTRODUCTIONINTRODUCTION Origins of remote sensing High resolution imaging sensors are very important in modern remote sensing technology. When pattern recognition methods are applied to remote sensing problems,  Smallness of the training data  Complex statistical distribution of a large number of classes

Map Asia 2003 M. İzzet SAĞLAM INTRODUCTIONINTRODUCTION The main purpose of developing a special classifier A new support vector learning algorithm The SOM-LSVMDT consists of  Clustering part  Binary tree structure with linear support vector machines in all tree nodes.

Map Asia 2003 M. İzzet SAĞLAM INTRODUCTIONINTRODUCTION The SOM-LSVMDT simplifies the model selection problem The SOM-LSVMDT has in-built properties for dealing with classes which can be considered as rare events. Reasons of occurrence of rare events.  Natural reason  SOM-LSVMDT’ structure To solve rare event problem

Map Asia 2003 M. İzzet SAĞLAM THE CLASSIFICATION PROBLEM Ten-class remote sensing problem.  Highly complex  Highly nonlinear problem space In addition, this work includes eight and thirteen class remote sensing problems of the same area.

Map Asia 2003 M. İzzet SAĞLAM 1. Problem Description The Colorado data set consists of 7 data channels obtained from the following 4 data sources:  Landsat MSS data (4 data channels)  Elevation data (in 10m contour intervals, 1data channel)  Slope data (0-90 degrees in degree increments, 1 data channel)  Aspect data (1-180 degrees in 1 degree increments, 1 data channel)

Map Asia 2003 M. İzzet SAĞLAM 1. Problem Description Table 1. Number of Samples, and the Ground Cover Classes of 10 Class Colorado Data Class FieldTestingTraining 1Water Colorado blue spruce2488 3Mountain/Subalpine meadow4245 4Aspen6575 5Ponderosa pine Ponderosa pine/Douglas fir Engelman spruce Douglas fir/White fir4432 9Douglas fir/ Ponderosa pine/Aspen25 10Douglas fir/White fir/Aspen3960

Map Asia 2003 M. İzzet SAĞLAM 2. Complexity Description Some of the classes are extremely under-represented.  Class 9  Class 1, 5, 6, 7 Highly nonlinear separation of the classes

Map Asia 2003 M. İzzet SAĞLAM LSVMDTLSVMDT LSVMDT includes binary tree structure with a linear SVM at each tree node (in the next Figure) All data vectors  Negative (output of the linear SVM  0)  Positive (output of the linear SVM > 0) Terms  Common class  Rare class  Class ratio

Map Asia 2003 M. İzzet SAĞLAM LSVMDTLSVMDT An example of a binary tree structure of the LSVM-DT.

Map Asia 2003 M. İzzet SAĞLAM A. Training Procedure 1.Train a linear SVM. After training is done, check a)lie on only one side of the decision hyperplane i.still lie on only one side of the decision hyperplane ii.else, store the linear SVM at the node currently pointed to by the pointer, and go to step 2 b)else, store the linear SVM at the node currently pointed to by the pointer, and go to step 2. 2.Separate all the vectors in the data set into two subsets

Map Asia 2003 M. İzzet SAĞLAM A. Training Procedure 3. For all vectors lying in the negative side a)if all of these vectors belong to the -1 class b)if not, create a new left child of the current tree node 4. For all vectors lying in the positive side a)if all of these vectors belong to the +1 class b)if not, create a new right child of the current tree node

Map Asia 2003 M. İzzet SAĞLAM B. Testing Procedure 1.Input the test vector to the linear SVM pointed to by the pointer 2.Check the output of the SVM a)if the output value is less than or equal to 0 b)if the output value is greater than 0 3.If the pointer points to a leaf node

Map Asia 2003 M. İzzet SAĞLAM THE PROPOSED ALGORITHM The SOM-LSVMDT consists of  Clustering part (in the following figure)  Binary tree structure with a linear SVM at each tree node (in the following figure)

Map Asia 2003 M. İzzet SAĞLAM THE PROPOSED ALGORITHM An example of a binary tree structure of the SOM-LSVMDT.

Map Asia 2003 M. İzzet SAĞLAM 1. Clustering Part The clustering part of SOM-LSVMDT  Initialization Random Sample Linear  Training

Map Asia 2003 M. İzzet SAĞLAM 1. Clustering Part The training procedure  The first stage is the winner node search  The second stage is adaptation

Map Asia 2003 M. İzzet SAĞLAM 1. Clustering Part Adaptation process in the clustering part

Map Asia 2003 M. İzzet SAĞLAM 2. LSVMDT Part In the secoond part of SOM-LSVMDT, each cluster generated by SOM is classified Rare class problem Total error of the proposed algorithm

Map Asia 2003 M. İzzet SAĞLAM EXPERIMENTAL RESULTS & CONCLUSIONS SOM-LSVMDT can achieve better performance than linear support vector machine decision tree (LSVMDT). The eight, ten, and thirteen Colorado data sets are used to obtain experimental results. The number of samples of each data set is showed in the following table.

Map Asia 2003 M. İzzet SAĞLAM EXPERIMENTAL RESULTS & CONCLUSIONS Table 2. Number of Samples each Colorado Data Set. 8-Class Colorado10-Class Colorado13-Class Colorado Training Test

Map Asia 2003 M. İzzet SAĞLAM EXPERIMENTAL RESULTS & CONCLUSIONS The results are shown in the following table. Repeated for three times Colorado data sets are clustered, the statistical distribution of the classes changes The most important point

Map Asia 2003 M. İzzet SAĞLAM EXPERIMENTAL RESULTS & CONCLUSIONS Table 3. Performance of the SOM-LSVMDT using the Colorado data sets. LSVMDTSOM-LSVMDT IISOM-LSVMDT IIISOM-LSVMDT IV 8 Class Colorado %7,17 ERROR%3,4 ERROR%0,97 ERROR%1,90 ERROR 10 Class Colorado %48,62 ERROR%35,02 ERROR%30,69 ERROR%35,98 ERROR 13 Class Colorado %29,48 ERROR%22,64 ERROR%20,08 ERROR%18,40 ERROR

Map Asia 2003 M. İzzet SAĞLAM THANK YOU M. Izzet SAGLAM * ITU INFORMATICS INSTITUE ADVANCED TECHNOLOGIES IN ENGINEERING * ITU SATELLITE COMMUNICATION AND REMOTE SENSING PROGRAM, Phd Student, INFORMATICS INSTITUE Research Assistant, ISTANBUL Prof. Dr. Bingul YAZGAN Prof. Dr. Okan ERSOY ITU SAGRES: