1. 1 Manifold Alignment for Multitemporal Hyperspectral Image Classification H. Lexie Yang 1, Melba M. Crawford 2 School of Civil Engineering, Purdue University and Laboratory for Applications of Remote Sensing Email: {hhyang 1, mcrawford 2 }@purdue.edu July 29, 2011 IEEE International Geoscience and Remote Sensing Symposium

2. 2 Outline Introduction Research Motivation −Effective exploitation of information for multitemporal classification in nonstationary environments −Goal: Learn “representative” data manifold Proposed Approach −Manifold alignment via given features −Manifold alignment via correspondences −Manifold alignment with spectral and spatial information Experimental Results Summary and Future Directions

3. 3 Introduction Challenges for classification of hyperspectral data −temporally nonstationary spectra −high dimensionality 2001200320042005200620022001 JuneJulyMay June N narrow spectral bands 1 2 3 N>>30

4. 4 Nonstationarities in sequence of images −Spectra of same class may evolve or drift over time Potential approaches −Semi-supervised methods −Adaptive schemes −Exploit similar data geometries  Explore data manifolds Research Motivation Good initial conditions required

5. 5 Manifold Learning for Hyperspectral Data Characterize data geometry with manifold learning −To capture nonlinear structures −To recover intrinsic space (preserve spectral neighbors) −To reduce data dimensionality Classification performed in low dimensional space Spectral bands Spatial dimension 1 2 3 4 2 nd dim 1 st dim 3 rd dim n 5 6 Original space Manifold space

6. 6 Challenges: Modeling Multitemporal Data Unfaithful joint manifold due to spectra shift Often difficult to model the inter-image correspondences Data manifold at T1 Data manifold at T2 Data manifolds at T 1 and T2

7. 7 Proposed Approach: Exploit Local Structure Assumption: local geometric structures are similar Approach: Extract and optimally align local geometry to minimize overall differences Locality Spectral space at T 1 Spectral space at T 2

8. 8 Proposed Approach: Conceptual Idea (Ham, 2005)

9. 9 Proposed Approach: Manifold Alignment Exploit labeled data for classification of multitemporal data sets Samples with no class labels Joint manifold Samples with class labels

10. 10 Manifold Alignment: Introduction and are 2 multitemporal hyperspectral images − Predict labels of using labeled Explore local geometries using graph Laplacian and some form of prior information Define Graph Laplacian −Two potential forms of prior information: given features and pairwise correspondences [Ham et al. 2005]

11. 11 Manifold Alignment via Given Features Given Features Joint Manifold Minimize

12. 12 Manifold Alignment via Pairwise Correspondences Correspondences between and Minimize Joint Manifold

13. 13 MA with spectral and spatial information Combine spatial locations with spectral signatures −To improve local geometries (spectral) quality −Idea: Increase similarity measure when two samples are close together Weight matrix for graph Laplacian: where spatial location of each pixel is represented as

14. 14 Experimental Results: Data Three Hyperion images collected in May, June and July 2001 May - June pair: Adjacent geographical area June - July pair: Targeted the same area May June July Class Water Floodplain Riparian Firescar Island interior Woodlands Savanna Short mopane Exposed soils

15. 15 Experimental Results: Framework Joint manifold Graph Laplacian Prior information Given features Correspondences Develop Data Manifold of Pooled Data Data setsLabels Pair 1 Pair 2 MayJuneTraining dataFor KNN classifier JuneJulyTesting dataFor overall accuracy evaluation Classification with KNN I 1, I 2 L I1I1 L I2I2 L

16. 16 Manifold Learning for Feature Extraction GlobalGlobal methods consider geodesic distance − Isometric feature mapping (ISOMAP) LocalLocal methods consider pairwise Euclidian distance −Locally Linear Embedding (LLE): (Saul and Roweis, 2000) −Local Tangent Space Alignment (LTSA): (Zhang and Zha, 2004) −Laplacian Eigenmaps (LE): (Belkin and Niyogi, 2004) (Tenenbaum, 2000)

17. 17 MA with Given Features Baseline: Joint manifold developed by pooled data (May, June pair) 79.21 77.29 77.88 76.31

18. 18 MA Results – Classification Accuracy Evaluate results by overall accuracies Methods Overall Accuracy May, JuneJune, July Manifold learning from pooled data62.38%83.00% Manifold alignment (MA) Given features (LE)79.21%86.16% Correspondences81.22%84.27% Methods Overall Accuracy May, JuneJune, July Given features (LE) Spectral79.21%86.16% Spectral + spatial84.21%90.30% Correspondences Spectral81.22%84.27% Spectral + spatial84.74%90.11%

19. 19 Results – Class Accuracy May/June Pair Typical class (Island Interior) Critical class (Woodlands) Critical class (Riparian) MA: Given Features: Spectral MA: Correspondences: Spectral MA: Given Features: Spectral / Spatial MA: Correspondences: Spectral / Spatial Pooled data

20. 20 Summary and Future Directions Multitemporal spectral changes result in failure to provide a faithful data manifold Manifold alignment framework demonstrates potential for nonstationary environment by utilizing similar local geometries and prior information Spatial proximity contributes to stabilization of local geometries for manifold alignment approaches Future directions −Investigate alternative spatial and spectral integration strategy −Address issue of longer sequences of images

21. 21 Thank you. Questions?

22. 22 References J. Ham, D. D. Lee, and L. K. Saul, “Semisupervised alignment of manifolds,” in International Workshop on Artificial Intelligence and Statistics, August 2005.

23. 23 Backup Slides