1. Remote Sensing Laboratory Dept. of Information Engineering and Computer Science University of Trento Via Sommarive, 14, I-38123 Povo, Trento, Italy Remote Sensing Laboratory Dept. of Information Engineering and Computer Science University of Trento Via Sommarive, 14, I-38123 Povo, Trento, Italy Begum Demir Francesca Bovolo Lorenzo Bruzzone Detection of Land-Cover Transitions in Multitemporal Remote Sensing Images with Active Learning Based Compound Classification Detection of Land-Cover Transitions in Multitemporal Remote Sensing Images with Active Learning Based Compound Classification E-mail: demir@disi.unitn.it Web page: http://rslab.disi.unitn.it

2. University of Trento, Italy Outline 2B. Demir, F. Bovolo, L. Bruzzone Introduction Aim of the work 1 Conclusion and future developments Proposed Joint Entropy based Active-Learning Method for Compound Classification 2 3 5 Experimental results 4

3. University of Trento, Italy 3 Detection of land-cover transitions between a pair of remote sensing images acquired on the same area at different times (i.e., multitemporal images) is very useful in many applications. Usually, this is achieved by supervised classification techniques, as unsupervised change detection methods have a reduced reliability in detecting explicitly different land-cover transitions. Such an approach requires ground reference data to detect changes and identify transitions. Due to the properties of the last generation of VHR passive sensors, supervised change-detection methods in real applications is becoming more and more important. Problem: The collection of a large multitemporal reference data is time consuming and expensive. Introduction B. Demir, F. Bovolo, L. Bruzzone

4. University of Trento, Italy 4 Goals Compute a map of land-cover transitions between a pair of remote sensing images acquired on the same area at different times. Take advantage of temporal dependence between images. Define a training set as small as possible. Assumptions The same set of land-cover classes characterizes the images. Initial training set with small number of labeled samples is available. Solution: Develop a novel Active Learning (AL) technique for compound classification of multitemporal remote sensing images that takes advantage of the temporal dependence among images. Aim of the Work B. Demir, F. Bovolo, L. Bruzzone

5. University of Trento, Italy 5B. Demir, F. Bovolo, L. Bruzzone G: Supervised classifier; Q: Query function; S: Supervisor; T: Training set; U: Unlabeled data I: Image Active Learning for Single Image Classification [1] S. Rajan, J. Ghosh, and M. M. Crawford, “An active learning approach to hyperspectral data classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 46, no. 4, pp. 1231-1242, Apr. 2008. [2] B. Demir, C. Persello, and L. Bruzzone, “Batch mode active learning methods for the interactive classification of remote sensing images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 49, no.3, pp. 1014-1031, March 2011. Update T G T G General AL Scheme Expanded T S Q U I

6. University of Trento, Italy Proposed Method: Block Scheme 6B. Demir, F. Bovolo, L. Bruzzone X1X1 X2X2 t 1 image Active Learning Compound Classifier t 2 image Map of land-cover transitions Different kinds of changes Training Set (pairs of temporally correlated labeled samples) Expanded Training Set

7. University of Trento, Italy Proposed Method: Block Scheme 7B. Demir, F. Bovolo, L. Bruzzone X1X1 X2X2 t 1 image Active Learning Compound Classifier t 2 image Map of land-cover transitions Different kinds of changes Training Set (pairs of temporally correlated labeled samples) Expanded Training Set

8. University of Trento, Italy Proposed Method: Compound Classifier 8B. Demir, F. Bovolo, L. Bruzzone X1X1 X2X2 t 1 image Compound Classifier Estimation of Classifier Parameters Training Set (pairs of temporally correlated labeled samples) L. Bruzzone, D. Fernandez Prieto, and S.B. Serpico, “A neural-statistical approach to multitemporal and multisource remote-sensing image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 37, no.3, pp. 1350-1359, May 1999. t 2 image Bayesian decision rule for compound classification: Number of classes Joint posterior probability Map of land-cover transitions Different kinds of changes

9. University of Trento, Italy Proposed Method: Compound Classifier 9 Assumption: class-conditional independence in the time domain B. Demir, F. Bovolo, L. Bruzzone Joint prior probability Joint class-conditional density Joint prior probabilities of land-cover transitions can be estimated on the basis of the expectation-maximization (EM) algorithm: Image size L. Bruzzone, D. Fernandez Prieto, and S.B. Serpico, “A neural-statistical approach to multitemporal and multisource remote-sensing image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 37, no.3, pp. 1350-1359, May 1999.

10. University of Trento, Italy Proposed Method: Block Scheme 10B. Demir, F. Bovolo, L. Bruzzone X1X1 X2X2 t 1 image Active Learning Compound Classifier t 2 image Map of land-cover transitions Different kinds of changes Training Set (pairs of temporally correlated labeled samples) Expanded Training Set

11. University of Trento, Italy 11B. Demir, F. Bovolo, L. Bruzzone X1X1 X2X2 t 1 image Joint Entropy Estimation of Statistical Distributions t 2 image Uncertain Samples Selection Update Training Set Proposed AL Procedure Expanded Training Set Joint prior probability Class conditional densities Proposed Method: Active Learning Joint Entropy No Yes Convergence? Joint prior probability

12. University of Trento, Italy 12B. Demir, F. Bovolo, L. Bruzzone We propose to use the joint entropy to measure the uncertainty: If joint entropy is small, the corresponding pair of pixels will be classified with high confidence, i.e., the decision on compound classification of these samples is reliable. If joint entropy is high, the decision is not reliable, and therefore the corresponding pair of samples is considered as uncertain and critical for the classifier. Joint entropy Joint posterior probability Proposed Method: Active Learning

13. University of Trento, Italy 13B. Demir, F. Bovolo, L. Bruzzone We adopted two possible simplifying assumptions that result in two different algorithms of the proposed AL technique: 1.Algorithm (JEAL) is defined under the assumption of class-conditional independence: Proposed Method: Active Learning Joint prior probability Class conditional densities

14. University of Trento, Italy 14B. Demir, F. Bovolo, L. Bruzzone 2.Algorithm (JEAL Ind ) is defined under the assumption of temporal independence: Proposed Method: Active Learning Marginal entropies the class-conditional independence the independence of a-priori class probabilities on the two images Joint prior probability Class conditional densities

15. University of Trento, Italy 15B. Demir, F. Bovolo, L. Bruzzone 1.Algorithm (JEAL) is defined under the assumption of class-conditional independence: 2.Algorithm (JEAL Ind ) is defined under the assumption of temporal independence: Proposed Method: Active Learning Mutual information

16. University of Trento, Italy Experimental Setup 16 B. Demir, F. Bovolo, L. Bruzzone Two different multitemporal and multispectral data sets are used (one made up of very high resolution images and one made up of medium resolution images). Class conditional densities are estimated from the available initial training set assuming Gaussian distribution. Results achieved with the proposed method are compared with Standard Marginal-Entropy based AL technique applied to the post classification comparison rule ignoring temporal dependence (Fully Independent).

17. University of Trento, Italy Data Set Description 17 Multitemporal data set: Two images acquired by the TM sensor of Landsat-5 satellite in September 1995 and July 1996 (Lake Mulargia, Sardinia Island, Italy). Land–cover classes: Pasture, Forest, Urban Area, Water, Vineyard September 1995 July 1996 B. Demir, F. Bovolo, L. Bruzzone Data SetNumber of Samples Pool2249 Test1949

18. University of Trento, Italy Experimental Results 18B. Demir, F. Bovolo, L. Bruzzone Proposed 1=Proposed JEAL method defined under the assumption of class-conditional independence. Proposed 2=Proposed JEAL method defined under the assumption of temporal independence.

19. University of Trento, Italy Experimental Results 19B. Demir, F. Bovolo, L. Bruzzone Proposed 1=Proposed JEAL method defined under the assumption of class-conditional independence. Fully Independent=Standard Marginal-Entropy based AL technique applied to the post classification comparison rule ignoring temporal dependence.

20. University of Trento, Italy Conclusion 20 A novel AL method has been defined on the basis of joint entropy defined in the context of compound classification for the detection of land-cover transitions. Two different joint entropy based AL algorithms are implemented under two possible simplifying assumptions: i) the class-conditional independence; and ii) the temporal independence between multitemporal images. Experiments show that the proposed joint entropy based AL technique, which takes advantage of temporal correlation, gives higher accuracies in detection of transitions. Proposed AL method  decreases significantly the cost and effort required for multitemporal reference data collection;  achieves high accuracy with a minimum number of multitemporal reference samples;  improves the performance of the standard marginal entropy based active learning method by exploiting temporal dependence between images. B. Demir, F. Bovolo, L. Bruzzone

21. University of Trento, Italy Future Development 21 Extend the proposed active learning algorithms  by including a diversity criterion defined in the context of compound classification.  considering label acquisition costs, which depend on locations and accessibility of the visited points for labeling the uncertain samples. B. Demir, F. Bovolo, L. Bruzzone