1. Abstract This poster presents results of three studies dealing with application of ARTMAP neural networks for classification of remotely sensed multispectral images. 1. Comparison of performance of ARTMAP classifiers with different types of cluster representation. In this study, the best results were obtained using Extended Gaussian ARTMAP. 2. A method for computation of classification accuracy index for Gaussian ARTMAP neural network. This method can be used to generate maps containing only pixels with prescribed minimum accuracy of classification. Methods for analysis and enhancement of neural network classification of remotely sensed images Norbert Kopčo, Peter Sinčák, and Rudolf Jakša Department of Cognitive and Neural Szstems, Boston University Computational Intelligence Group, Technical University of Košice, Slovakia

2. 3. Preliminary results obtained on a hierarchical ARTMAP architecture based on set of dichotomical classifiers. This neural network is suitable for parallel processing of large data sets. Refrences Sinčák, P., Veregin, H., and Kopčo, N.: Computational intelligence for classification of remotely sensed images. Neural Network World, v. 5, 1998, pp. 577-593. Sinčák, P., Kopčo, N., and Veregin H. (unpublished) Conflation Techniques to Improve Image Classification Accuracy. Submitted to Photogrametric Engineering and Remote Sensing. Carpenter, G. and Grossberg, S. (1992) Fuzzy ARTMAP: A Neural Network Architecture for Incremental Supervised Learning of Analog Multidimensional Maps. IEEE Trans. on Neural Networks, 3: 197-214. Williamson, J.R. (1996) Gaussian ARTMAP: A Neural Network for Fast Incremental Learning of Noisy Multidimensional Maps, Neural Networks, pp. 881-897. Cunningham, R. K. (1998) Learning and recognizing patterns of visual motion, color, and form. Unpublished Ph.D. thesis. Boston University.

3. 1. Evaluation of dependence on cluster representation in ARTMAP classifiers There is a large number of classification algorithms available without detailed knowledge of their properties/performance. Neural networks are considered to be assumption-less classifiers (for a successful classification they do not require explicit assumptions about the data distribution). But these systems have implicit assumptions built into them. These assumptions are related to the data representation and algorithmic properties of the system. In the present study, the dependence of the ARTMAP classifiers performance on the internal cluster representation is analyzed for image data from remote sensing.

4. Data-set Seven-dimensional Landsat TM image of the city of Košice (Figure 1) Size of image: 368,125 pixels, out of which 6,331 classified by an expert into seven categories (A - urban area, B - barren fields, C - bushes, D - agricultural fields, E - Meadows, F - Woods, and G - water) Method of analysis The performance is compared in terms of weighted PCC (Percent of Correctly Classified) and the contingency tables. Compared systems Fuzzy ARTMAP - FA (Carpenter et al., 1992) Gaussian ARTMAP - GA (Williamson, 1996) Extended Gaussian ARTMAP - EGA (Cunningham, 1997)

5. Figure 1: Original image

6. ARTMAP classifier topology Input OutputMapField (Labeling) Recognition (Clustering) Comparison Cluster representation: Fuzzy ARTMAP: Hyper-rectangles Gaussian ARTMAP: Gaussian distributions without covariance Extended Gaussian ARTMAP: Gaussian distribution with covariance F0F1F2MFOL FA EGA GA

7. Results Weighted PCC for five permutations of the training set and voting Gaussian distribution more suitable as representation of the clusters in image classification tasks (see Fig. 2) Best performance achieved by Extended Gaussian ARTMAP (although differences not very significant) Contingency table for Extended Gaussian ARTMAP Predicted class

8. Figure 2: Image classified by Extended Gaussian ARTMAP Sensitivity to the ordering of the training set smaller for GA and EGA than for the FA During learning, FA always reached 100% accuracy. GA/EGA accuracy was around 97.7%, which means stronger generalization.

9. 2. Confidence Index In remote sensing, a large amount of data is produced, which needs to be quickly and reliably classified. Often, explicit assessment of the reliability is needed. Goals: Develop a method for simple and fast assessment of the accuracy of classification. For the accuracy assessment, exploit computations done during the classification process. Develop the assessment method for the Gaussian ARTMAP algorithm, which achieved the best performance in the previous study.

10. Method GA uses Bayes discrimination function: (1) The pattern is classified into the category with the largest probability measure: (2) If voting is used, the probability measures are evaluated over all the networks: (3) The confidence index is then defined as: (4)

11. Figure 3: Confidence map for Gaussian ARTMAP classification (expressed in %)

12. Confidence threshold A confidence threshold can be defined, which, in combination with the confidence index, can be used to generate maps with arbitrary accuracy. Two counteracting aspects of classification are influenced by the choice of threshold: 1) accuracy of classification, and 2) number of unclassified patterns (see graph below)

13. Figure 4: Thresholded classification map for Gaussian ARTMAP (accuracy 99%,  =0.921)

14. Results and discussion The presented method offers a simple and fast way for assessment of quality of the Gaussian ARTMAP neural network classification. The method can be used also with other neural networks employing the voting strategy. The method offers a tool for production of maps with arbitrary classification accuracy. Because of the way in which this method exploits the computations done during the classification, it is possible that the confidence assigned to some of the classified points is incorrect.

15. Often, the complexity of the data is too high, or the size of the data set is too large (in terms of memory or time requirements), for standard versions of classifiers. To overcome this problem, hierarchical/modular classification systems are usually applied. Here, a hierarchical structure based on ARTMAP networks is proposed, and its properties are analyzed. The system is called Parallel ARTMAP, because it consists of ART sub-nets, each trained independently on a single category (i.e., each sub-net learns to detect data from one category). In the following simulations, fuzzy ART/ARTMAP system is used in the sub-net modules of the network. 3. Hierarchical classifier

16. Parallel processing system Every sub-net in the clustering layer is trained on data belonging to a single class, i.e., it learns to detect data from that class. Conflict resolution module serves to resolve conflicts if two or more clustering nets identify a pattern as “theirs”. Two design choices: 1) rule for determination of optimal  for each clustering subnet, 2) conflict-resolving rule. ART Parallel ARTMAP system Clustering subsystem (class detectors) Conflict resolution Class 1 data Class 2 data Class 3 data Class n data ART

17. Optimal  determination rule Each sub-net is trained on patterns from a single class using cross-validation: - 9/10 - estimation set, - 1/10 - validation set. Training is repeated for different values of . Rule: Choose  which corresponds to the the point just before the first dip in the “true positive” graph. This rule assures that each detector will correctly identify almost all of the patterns belonging to it, while minimizing the false alarm rate. 0.70.80.91 0 0.2 0.4 0.6 0.8 1 Graph of performance of clustering sub-net for class #7 Parameter  Percent of correctly classified Other classes True “–” rate My class (#7) True “+” rate Optimal 

18. 0.70.80.91 0 0.2 0.4 0.6 0.8 1 Class 1  PCC 0.70.80.91 0 0.2 0.4 0.6 0.8 1 Class 2  PCC 0.70.80.91 0 0.2 0.4 0.6 0.8 1 Class 3  PCC 0.70.80.91 0 0.2 0.4 0.6 0.8 1 Class 4  PCC 0.70.80.91 0 0.2 0.4 0.6 0.8 1 Class 5  PCC 0.70.80.91 0 0.2 0.4 0.6 0.8 1 Class 6  PCC 0.70.80.91 0 0.2 0.4 0.6 0.8 1 Class 7  PCC Optimal  for all sub-nets Other classes True negative rate My class True positive rate Optimal 

19. For the previously defined  -determination rule, all the sub-nets will have almost 100% true positive rate. But their false alarm rate (false negative rate) will be non-zero. So the nets are biased towards identifying patterns as “theirs”. Moreover, the false alarm rate will be different for each sub-net. So the conflict resolving rule can be based on the true negative rate. Conflict-resolving rule Rule: If there is a conflict between two or more sub-nets, assign the pattern to the class with the largest true negative rate. This rule was chosen, because the parameters needed for conflict resolution (true “-” rates) are easily computed during the  -determination process.

20. Results and discussion Overall classification accuracy is 78.89%. The decreased performance is caused mainly by the misclassification of the patterns from classes C and F. Despite worse performance, the system can be usefully applied to large data sets. Also the speed of the system can be advantageous. Choice of alternative rules for  -determination and conflict resolution can improve the performance. Predicted class Contingency table for Parallel ARTMAP