1. Laboratory for Applications of Remote Sensing Critical Class Oriented Active Learning for Hyperspectral Image Classification Hyperspectral Image Classification School of Civil Engineering, Purdue University and Laboratory for Applications of Remote Sensing Email: {wdi@purdue.edu 1, mcrawford 2 }@purdue.edu July 28, 2011 IEEE International Geoscience and Remote Sensing Symposium Wei Di and Melba Crawford

2. Outline  Background Critical Class Oriented Active Learning(AL)  Proposed Methods (SVM-CC, SVM-CC MS ) –Guided & Active Learning –Critical Class Oriented –Margin Sampling Based  Experimental Results  Conclusions & Future Work

3. Laboratory for Applications of Remote Sensing I. BACKGROUND

4. Motivation Sampling Strategy D L Pool Training Data Target H Supervised Classifier Achieve better performance Higher utility, low redundancy Economically allocate resources for labeling Focus on a specific task or requirement Intelligent sampling strategy

5. Active Learning Query Strategy D L Pool D U Pool Supervised Classifier ClassifierSupervised New x L Output Classifier Training xUxU f(x u ) Passive Learning Active Learning (AL) - Active Learning (AL) - Iterative learning circle Uncertainty & Critical Class

6. Introduction  Active Learning in remote sensing -Classification: Tuia et al. [2009], Patra and Bruzzone [2011] Demir et al. [2011], Di and Crawford [2011],. -Segmentation: Jun et al. [2010]  Critical Class oriented Active Learning - Shifting hyperplane by pair-wise SVM -Identify “Difficult” Classes -Category based query & margin sampling  Goal Provide concept level guidance for building training set favoring “difficult” classes

7. Laboratory for Applications of Remote Sensing II. PROPOSED METHOD

8. Key Idea: Shifting Hyperplane Pair-wise Class A and B Class A Class B Hyperplane w Margin Support Vectors Hyperplane Margin New Samples Cumulative Change Critical Significant Level Changing Hyperplane

9. Critical Class Identification  Query-based Regularizer w k - hyperplane vector by SVM for kth binary class at the  th query.  Accumulated Margin Instability  Measure the cumulative changes  Order Statistic  Rank class pairs:  Prob. of the kth class pair at critical level C L :

10. Critical Class Query Critical Class Set Critical Class Pair Higher probability Critical Class Identification SVM-CC SVM-CC Random Query From Critical Class Set SVM-CC MS SVM-CC MS Query Sample within Critical Class set and closest to margin Critical Class Set  Query

11. Laboratory for Applications of Remote Sensing III. EXPERIMENTAL RESULTS

12. Kennedy Space Center & Botswana Data AVIRIS hyperspectral data Acquired on March, 1996 176 of total 224 bands Spectral bandwidth 10nm Spatial resolution 18m Data Description KSCBOT C LASS N AME No.C LASS N AME No. 1 Scrub761Water361 2 Willow Swamp243Primary Floodplain308 3 Cabbage Palm Hammock*256Riparian*303 4 Cabbage Palm/Oak Hammock* 252Firescar335 5 Slash Pine*161Island Interior370 6 Oak / Broadleaf hammock*229Woodlands*324 7 Hardwood Swamp*105Savanna342 8 Graminoid Marsh431Short Mopane299 9 Salt Marsh419Exposed Soils229 10 Water927 * Denotes the hard classes

13. Experimental Results IndexClass-Pair KSC 18  (3,4)  Cabbage Palm Hammock;  Cabbage Palm/Oak Hammock 26  (4,6)  Cabbage Palm/Oak Hammock  Oak / Broadleaf hammock BOT18  (3,6)  Riparian  Woodlands Accumulated Margin Instability (AMI) 10 th 30 th AMI as learning process KSC BOT

14. Experimental Results DTDT KSC at 600 th queryBOT at 400 th query Class Index CCCC MS SVM MS CCCC MS SVM MS C1-0.37-0.41 -0.27 000 C22.573.97 1.95 -0.070.821.10 C3-1.87-0.80 -2.02 11.6412.049.08 C47.339.244.84-0.24-0.12-0.06 C51.334.00 2.11 -5.00-1.76-1.91 C62.646.34 2.71 3.683.07-0.43 C74.790.30 7.88 0.672.502.32 C8-3.86-2.97 -3.20 0.330.600.66 C9-0.28-0.72 -0.08 -1.06-1.68-0.27 C1000 0 DTDT DUDU Learning Curve Per-Class Improvement vs RS

15. Experimental Results KSCC1C2C3C4C5C6C7C8C9C10 CC0.600.300.730.830.850.740.450.250.240.27 CC MS 0.500.490.80.890.940.780.380.320.210.23 SVM MS 0.350.600.360.580.670.540.450.540.400.44 RS0.480.43 0.450.480.450.420.450.490.45 BOTC1C2C3C4C5C6C7C8C9 CC0.030.310.890.110.040.880.130.040.13 CC MS 00.190.950.070.100.950.150.080.04 SVM MS 0.200.230.390.360.220.440.280.180.20 RS0.280.29 0.300.280.260.270.250.29 Per-Class Sampling Ratio Per-class Sampling Ratio Ratio of Support Vectors RS SVM MS CC CC MS KSC

16. Laboratory for Applications of Remote Sensing IV. CONCLUSIONS AND FUTURE WORK

17. Conclusions & Future Work  Conclusions -Shifting Hyperplane – Provides valuable information for identifying difficult classes. -Critical Class Oriented Margin Sampling – Focuses on difficult classes, as well as informative samples, improve performance in multi-class problem. -Support Vectors - Concentrate on samples likely to be support vectors.  Future work -Investigation of feature subspaces for identifying the critical classes. -Design proper sample-wise utility score to enhance the category based query.

18. Laboratory for Applications of Remote Sensing IV. CONCLUSIONS AND FUTURE WORK

19. Conclusions & Future Work  Conclusions -Shifting Hyperplane – Provides valuable information for identifying difficult classes. -Critical Class Oriented Margin Sampling – Focuses on difficult classes, as well as informative samples; improves performance in multi-class problem. -Support Vectors - Concentrate on samples likely to be support vectors.  Future work -Investigation of the feature subspace for identifying the critical classes. -Design proper sample-wise utility score to enhance the category based query.

20. Thanks very much!

21. Critical Class Identification Process Accumulative Margin Instability Critical Class Probability Heat Map

22. Experimental Results (a) KSC: RS (b) KSC: SVM MS (c) KSC: SVM-CC (d) KSC: SVM-CC MS Per-class Learning Performance

23. Experimental Results RS SVM MS SVM-CC SVM-CC MS BOT Ratio of Support Vectors