1. Know the Earth…Show the Way NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Approved for Public Release 05-269 Hyperspectral Imagery (HSI) Dimensionality Reduction Ronald G. Resmini, Ph.D. 18 July 2005 Institute for Pure and Applied Mathematics (IPAM) v: 703-735-3899 ronald.g.resminir@nga.mil

2. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 2 Approved for Public Release 05-269 Outline Introduction to HSI HSI Dimensionality HSI Dimensionality Reduction (DR)  Do We Need DR? HSI Algorithms What Can You Do? How Should You Do It? Introduction to HSI HSI Dimensionality HSI Dimensionality Reduction (DR)  Do We Need DR? HSI Algorithms What Can You Do? How Should You Do It?

3. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 3 Approved for Public Release 05-269 writ large...the phenomenology of spectra; remote material detection, identification, characterization and quantification Introduction to Hyperspectral Imagery (HSI) Remote Sensing HSI is, fundamentally:

4. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 4 Approved for Public Release 05-269 HSI Remote Sensing: Frame of Reference... Remote sensing of the earth  airborne  spaceborne  ground (portables) But bear in mind other apps:  medical  industrial  many, many others

5. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 5 Approved for Public Release 05-269 Electromagnetic Energy Electromagnetic Spectrum Electromagnetic Spectrum Wavelength (nm) Cosmic Rays Gamma Rays X Rays Microwaves (Radar) Radio & Television Waves UV 10 5 10 6 10 7 10 8 10 9 10 10 10 11 10 12 10 1 10 10 -1 10 -2 10 -3 10 -4 10 -5 Shorter Wavelengths High Energy Shorter Wavelengths High Energy Longer Wavelengths Low Energy Longer Wavelengths Low Energy V / NIR / SWIR / MWIR / LWIR Optical Region 400 14000 400 0.4 400 0.4 14000 14.0 14000 14.0 1500 1.5 1500 1.5 3000 3.0 3000 3.0 5000 5.0 5000 5.0 700 0.7 700 0.7 NIR MWIR SWIR R G LWIR B LWIR Wavelength (nm) (  m) Wavelength (nm) (  m) Emitted Energy Reflected Energy

6. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 6 Approved for Public Release 05-269 Reflected vs. Emitted Energy 1 10 4 1000 100 10 0.1 1105 3 7 Irradiance (W-m -2 -um -1 ) Wavelength (µm) Earth Emission (100%) Earth Reflectance (100%) radiant exitance (W-m -2 -um -1 ) MWIR Assumes no atmosphere.4.7

7. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 7 Approved for Public Release 05-269 Sampling the Spectrum NIR SWIR MWIR LWIR 400 nm 700 15003000 RB 500014000 nm G Panchromatic : one very wide band LOW Multispectral : several to tens of bands MED Hyperspectral : hundreds of narrow bands HIGH

8. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 8 Approved for Public Release 05-269 Interaction of energy and objects Transmitted Energy Absorbed Energy Reflected Energy V-MWIR Emitted Energy MW-LWIR Energy Balance Equation: E I ( ) = E R ( ) + E A ( ) + E T ( ) Incident Energy

9. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 9 Approved for Public Release 05-269 NASA AVIRIS Cuprite, NV, HSI Data, (1995)

10. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 10 Approved for Public Release 05-269 An AVIRIS (NASA) HSI Image Cube

11. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 11 Approved for Public Release 05-269 The Spectrum is the Fundamental Datum of HSI RS

12. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 12 Approved for Public Release 05-269 Applications of HSI RS Geology Forestry Agriculture Mapping/land use, land cover analysis Atmospheric analysis Environmental monitoring Littoral zone RS Many, many others

13. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 13 Approved for Public Release 05-269 Levels of Spectral Information Quantification: Determines the abundance of materials. Characterization: Determines variability of identified material (e.g. wet/dry sand, soil particle size effects). Identification: Determines the unique identity of the foregoing generic categories (i.e. material identification). Discrimination: Determines generic categories of the foregoing classes. Classification: Separates materials into spectrally similar groups. Detection: Determines the presence of materials, objects, activities, or events. Quantification: Determines the abundance of materials. Characterization: Determines variability of identified material (e.g. wet/dry sand, soil particle size effects). Identification: Determines the unique identity of the foregoing generic categories (i.e. material identification). Discrimination: Determines generic categories of the foregoing classes. Classification: Separates materials into spectrally similar groups. Detection: Determines the presence of materials, objects, activities, or events. Panchromatic Low Spectral Resolution High Spectral Resolution Hyperspectral (100’s of bands) Hyperspectral (100’s of bands) Multispectral (10’s of bands) Multispectral (10’s of bands)

14. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 14 Approved for Public Release 05-269 Image from the NASA Langley Research Center, Atmospheric Sciences Division. http://asd-www.larc.nasa.gov/erbe/ASDerbe.html

15. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 15 Approved for Public Release 05-269 Electromagnetic Energy Atmospheric Absorption

16. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 16 Approved for Public Release 05-269 Reflectance: Is the ratio of reflected energy to incident energy.   Varies with wavelength   Function of the molecular properties of the material. Reflectance Signature: A plot of the reflectance of a material as a function of wavelength. Reflectance: Is the ratio of reflected energy to incident energy.   Varies with wavelength   Function of the molecular properties of the material. Reflectance Signature: A plot of the reflectance of a material as a function of wavelength. Reflected Energy Red brick Kaolinite Sandy loam Concrete Grass All solids and liquids have reflectance signatures that potentially can be used to identify them.

17. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 17 Approved for Public Release 05-269 Emissive Energy Basic Concepts Blackbody – A theoretical material that absorbs and radiates 100% of the energy incident upon it. BB curve is a function of temperature and wavelength. Planck’s Law – gives shape of blackbody curve at a specific temperature. Wien’s Displacement Law – determines wavelength of peak emittance. Blackbody – A theoretical material that absorbs and radiates 100% of the energy incident upon it. BB curve is a function of temperature and wavelength. Planck’s Law – gives shape of blackbody curve at a specific temperature. Wien’s Displacement Law – determines wavelength of peak emittance. Wavelength (µm) 0.2 0.4 0.7 1 2 3 5 8 10 30 Spectral Radiant Emittance Peak Emittance 300K Ambient 250K 500K 800K 373K Boiling Water 6000K Sun 3000K Light Bulb 1500K Hot Coals

18. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 18 Approved for Public Release 05-269 The Planck or Blackbody Radiation Equation: Units:

19. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 19 Approved for Public Release 05-269 Emissive Energy Emissivity - is a measure of how efficiently an object radiates energy compared to a blackbody at the same temperature.   Varies with wavelength   Function of the molecular properties of the material. Emissivity Signature - A plot of emissivity as a function of wavelength. All materials have emissivity signatures that potentially can be used to identify them. Emissivity - is a measure of how efficiently an object radiates energy compared to a blackbody at the same temperature.   Varies with wavelength   Function of the molecular properties of the material. Emissivity Signature - A plot of emissivity as a function of wavelength. All materials have emissivity signatures that potentially can be used to identify them. Blackbody Graybody Selective emitter (emissivity signature) Selective emitter (emissivity signature) Emissivity 0 0.5 1.0 Wavelength Red brick Kaolinite Grass Water Black paint Concrete

20. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 20 Approved for Public Release 05-269 Spectral Signature Libraries Spectral signatures of thousands of materials (solid, liquid, gas) have been measured in the laboratory and gathered into “libraries”. Library signatures are used as the basis for identification of materials in HSI data. Spectral signatures of thousands of materials (solid, liquid, gas) have been measured in the laboratory and gathered into “libraries”. Library signatures are used as the basis for identification of materials in HSI data. (...beyond scope for a discussion on DR; but...)

21. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 21 Approved for Public Release 05-269 Understanding Spectral Data: Signature Variability Factors  Brightness  BRDF  Target morphology shape orientation  Particle size  Moisture  Spectral mixing  Brightness  BRDF  Target morphology shape orientation  Particle size  Moisture  Spectral mixing  Composition original change over time  Surface quality roughness weathering  Shade & Shadow  Temperature

22. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 22 Approved for Public Release 05-269 Reflected Energy The manner in which a material reflects energy is primarily a function of the optical properties and surface roughness of the feature. Most objects are diffuse reflectors The manner in which a material reflects energy is primarily a function of the optical properties and surface roughness of the feature. Most objects are diffuse reflectors Specular Reflectance Diffuse Reflectance Angle of Incidence = Angle of Reflectance Smooth Surface Rough Surface (Microscopic) Energy Scattered in All Directions

23. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 23 Approved for Public Release 05-269 Emissive Energy Identification of Gases Detected Signature Plume Wavelength Emission Background (Cool) Gas (Warm) Gases appear in either emission or absorption depending on the temperature contrast between the gas and the background. Same Temperature Wavelength No Detection Background Gas Wavelength Absorption Background (Warm) Gas (Cool)

24. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 24 Approved for Public Release 05-269 Stefan-Boltzmann Law: Two surfaces radiating at each other: View Factor Algebra and Radiant Exchange......from Welty, Wicks, and Wilson (1984)

25. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 25 Approved for Public Release 05-269 Spectral Mixture Analysis (SMA) An area of ground of, say 1.5 m by 1.5 m may contain 3 materials: A, B, and C. An HSI sensor with a GSD of 1.5 m would measure the ‘Mixture’ spectrum SMA is an inversion technique to determine the quantities of A, B, and C in the ‘Mixture’ spectrum SMA is physically-based on the spectral interaction of photons of light and matter SMA is in widespread use today in all sectors utilizing spectral remote sensing Variations include different constraints on the inversion; linear SMA; nonlinear SMA ‘Mixture’ = 25%A + 35%B + 40%C

26. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 26 Approved for Public Release 05-269 Spatial Resolution Radiometric Resolution Temporal Resolution Spatial Resolution Radiometric Resolution Temporal Resolution Resolutions (...beyond scope for a discussion on DR; but...)

27. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 27 Approved for Public Release 05-269 HSI Fundamentals Summary Hyperspectral remote sensing involves measuring energy in the Visible – LWIR portions of the electromagnetic spectrum. Some of the measured energy is reflected from objects while some energy is emitted from objects. Every material has a unique spectral signature. Spectral image data are collected such that signatures can be extracted for material detection, classification, identification, characterization, and quantification. Spectral, spatial, radiometric, and temporal resolution determine the capabilities of the remote sensing sensor/system. Hyperspectral remote sensing involves measuring energy in the Visible – LWIR portions of the electromagnetic spectrum. Some of the measured energy is reflected from objects while some energy is emitted from objects. Every material has a unique spectral signature. Spectral image data are collected such that signatures can be extracted for material detection, classification, identification, characterization, and quantification. Spectral, spatial, radiometric, and temporal resolution determine the capabilities of the remote sensing sensor/system.

28. The General Data Analysis/Exploitation Flow DN Calibration Fixes/Corrections Data Ingest Look At/Inspect the Data!! Atmospheric Compensation Algorithms for Information Extraction Information Fusion Geometric/Geospatial Product/Report Generation Distribution Archive/Dissemination Planning for Additional Collections Spectral Library Access Iteration DR? Approved for Public Release 05-269

29. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 29 Approved for Public Release 05-269 HSI Remote Sensing: Frame of Reference... A Scientist’s Approach to the Data:  look at the data(!)  observables have a physical, chemical, biological, etc. basis  must understand nature of observables  bumps and wiggles have real, physical (spectroscopic) significance  application of tools comes last!

30. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 30 Approved for Public Release 05-269 Defining HSI Dimensionality Hundreds of bands of data in an HSI data cube An HSI pixel (a spectrum) is an n-D vector  n = number of bands  a spectrum is a point in an n-D space “Redundancy” of information Embedding or spanning dimension Intrinsic dimension/virtual dimension A distinction  large volume of data  dimensionality

31. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 31 Approved for Public Release 05-269 The n-D Space — Where Many Algorithms Operate Each HSI spectrum (or pixel) is an n-D vector that can be represented as a single point in n-D space. n-D space is actually where many of our algorithms operate. Wavelength (  m) Reflectivity, 

32. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 32 Approved for Public Release 05-269 Four (A-D) Equivalent Notations/Representations Wavelength (micrometers) Reflectance,  (0.11, 0.23, 0.30, 0.25, 0.16, 0.27, 0.31, 0.37,...,) , Band a , Band b Spectrum s 1...imagine an n-D hyperspace... A B C D

33. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 33 Approved for Public Release 05-269 Some HSI Scatter Plots; Spectra as Points in ‘Hyperspace’

34. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 34 Approved for Public Release 05-269 Defining HSI Dimensionality “Curse of dimensionality”  for Gaussian distribution... ...for a given classification accuracy  # of training samples grows quadratically  based on exploitation methodology; e.g.: Mahalanobis Distance: Maximum Likelihood:

35. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 35 Approved for Public Release 05-269 HSI or MSI 100’s of bands vs. 10’s of bands Maybe all you need is 6 bands but...  you need six; and you need six; and so on Atmospheric compensation... HSI is spectroscopy writ large  its about resolving spectral information  fine spectral features  broad spectral features Today’s FPAs make HSI a breeze anyway...

36. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 36 Approved for Public Release 05-269 Multispectral - Hyperspectral Signature Comparison Multispectral Hyperspectral Resampled to Landsat TM7 Bands

37. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 37 Approved for Public Release 05-269 400 0.40 400 0.40 1500 1.50 1500 1.50 3000 3.00 3000 3.00 700 0.70 700 0.70 NIR SWIR RGB Wavelength (nm) (  m) Wavelength (nm) (  m) Minerals/Geology Soils Bathymetry Vegetation Fuels Aerosols Atmos. Comp. Plastics Fabrics Paints O2O2 CO 2 Chlorophyll DOM/CDOM Cirrus Iron oxides Similar figures may be constructed for M/LWIR regions.

38. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 38 Approved for Public Release 05-269 Estimating HSI Dimensionality Eigenvalues of the covariance  principal components analysis (PCA)/aka KL  optimal, least squares sense Eigenvalues of the correlation matrix Visual—based on eigenvalues Continuous significant linear dimensionality  CSD; eigenvalues (next slide...) Umaña-Díaz, A., and Vélez-Reyes, M., (2003). Determining the dimensionality of hyperspectral imagery for unsupervised band selection. Proceedings of the SPIE, S.S Shen and P.E. Lewis, eds., v. 5093, pp. 70-81. (...and references cited therein.)

39. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 39 Approved for Public Release 05-269 where: i are the eigenvalues Covariance Matrix: Find the eigenvalues of the covariance (or correlation) matrix and then... or:

40. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 40 Approved for Public Release 05-269 Band Number Eigenvalue Virginia City Probe-1 HSI Data Eigenvalues from a PCA 116 bands out of 128

41. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 41 Approved for Public Release 05-269 Band Number Eigenvalue Eigenvalues from a PCA 162 bands out of 210 Urban Scene HYDICE HSI Data

42. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 42 Approved for Public Release 05-269 Band Number Eigenvalue Eigenvalues from a PCA 128 bands out of 128 Mormon Mesa SEBASS HSI Data

43. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 43 Approved for Public Release 05-269 Estimating HSI Dimensionality Wavelet basis Nonlinear dimension estimation  Near neighbor method of Pettis  Fukunaga and Olsen’s KL-related method  Fractal dimension o Hausdorff dimension o Box-counting method o Correlation integral/dimension (next slide...)

44. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 44 Approved for Public Release 05-269 Box Counting: Correlation Dimension: where: r is box size (D B ) or radius of a hypersphere (D) Correlation Function:

45. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 45 Approved for Public Release 05-269 HSI Dimensionality Reduction Techniques  Principal components analysis (PCA)  Minimum noise fraction (MNF)  Vector quantization (VQ)  Projection pursuit (PP)  The universe of data compression o lossless/lossy (when/why?) o discrete cosine transformation (DCT) o wavelets-based compression  Best bands selection/band averaging

46. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 46 Approved for Public Release 05-269 Best-Bands Selection 2.0  m to 2.5  m – SWIR, Only

47. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 47 Approved for Public Release 05-269 Other Means of DR Spectral mixture analysis (SMA)  basis vectors Analysis of filter vectors (OSP algorithms...) Wavelet-based feature selection On-board processing  transmit product  advanced computation  quantum computation?

48. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 48 Approved for Public Release 05-269 Other Means of DR Transmit only bands of interest  best-bands selection/band averaging... ...perhaps after atmos. comp. Spectral parameterizations Derivative spectroscopy Binary encoding

49. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 49 Approved for Public Release 05-269 Spectral Parameterization: Spectral Metrics (1 of 2) Soil Spectrum

50. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 50 Approved for Public Release 05-269 Desert Soil (Malpais) 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.000.100.200.300.400.50 3.5 - 4.17 microns Band 2 FWHM 3.5 - 4.17 microns Band 1 Depth Disturbed Soil Pristine Soil Vehicle Treads Spectral Parameterization: Spectral Metrics (2 of 2)

51. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 51 Approved for Public Release 05-269 The Need For DR Is there a need for DR? Is there a “curse of dimensionality”? Well...it depends... Not with today’s (and tomorrow’s) computers Not with many capable HSI algorithms Structure of HSI in n-D space  linear mixing trends  mixed pixels; spectral endmembers  are these clusters? Yes...if using traditional MSI classification techniques...

52. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 52 Approved for Public Release 05-269 Mahalanobis Distance Maximum Likelihood

53. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 53 Approved for Public Release 05-269 Jeffries-Matusita (JM) Distance Where B is the Bhattacharyya distance

54. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 54 Approved for Public Release 05-269 HSI Algorithms

55. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 55 Approved for Public Release 05-269 Euclidean Distance: n-D Geometry A 2D scatterplot with 2 spectra: Band a Band b Spectrum s 1 Spectrum s 2 Whole-Pixel Distance Metric in n-D Hyperspace Assume a two band spectral remote sensing system. Each two point ‘spectrum’ is a point in Band b vs. Band a space. Euclidean Distance

56. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 56 Approved for Public Release 05-269 SAM: n-D Geometry A 2D scatterplot with 2 spectra: Band a Band b Spectrum s 1 Spectrum s 2  Angular Distance Metric (Spectral Angle Mapper or SAM) Assume a two band spectral remote sensing system. Each two point ‘spectrum’ is a point in Band b vs. Band a space. The angle, , between the two lines connecting each spectrum (point) to the origin is the angular separation of the two spectra. Smaller angular separations in- dicate more similar spectra.

57. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 57 Approved for Public Release 05-269 SAM: The Math Chang (2003), ch. 2, pp. 20-21; and... Assume two 5-band spectra as shown:

58. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 58 Approved for Public Release 05-269 Linear Spectral Unmixing The reflectance of an image pixel is a linear combination of reflectances from (typically) several “pure” substances (or endmembers) contained within the ground-spot sampled by the remote sensing system: where: R i is the reflectance of a pixel in band i, f j is the fractional abundance of endmember j in the pixel, M j,i is the reflectance of endmember substance j in band i, r i is the unmodeled reflectance for the pixel in band i, and n is the number of endmembers.

59. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 59 Approved for Public Release 05-269 A linear equation... 7 x 5 5 x 1 7 x 1 5 endmembers in a 7-band spectral data set A x b

60. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 60 Approved for Public Release 05-269 OSP/LPD/DSR: Scene-Derived Endmembers (Harsanyi et al., 1994; see also ch. 3 of Chang, 2003)

61. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 61 Approved for Public Release 05-269

62. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 62 Approved for Public Release 05-269

63. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 63 Approved for Public Release 05-269 This is equivalent to Unconstrained SMA The value of x T which maximizes is given by x T = d T

64. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 64 Approved for Public Release 05-269 Statistical Characterization of the Background (LPD/DSR) (Harsanyi et al., 1994)

65. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 65 Approved for Public Release 05-269

66. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 66 Approved for Public Release 05-269 Constrained Energy Minimization (CEM) The description of CEM is similar to that of OSP/DSR (previous slides) Like OSP and DSR, CEM is an Orthogonal Subspace Projection (OSP) family algorithm CEM differs from OSP/DSR in the following, important ways:  CEM does not simply project away the first n eigenvectors  The CEM operator is built using a weighted combination of the eigenvectors (all or a subset) Though an OSP algorithm, the structure of CEM is equally readily observed by a formal derivation using a Lagrange multiplier CEM is a commonly used statistical spectral matched filter CEM for spectral remote sensing has been published on for over 10 years CEM has a much longer history in the multi-dimensional/array signal processing literature Just about all HSI tools today contain CEM or a variant of CEM If an algorithm is using M -1 d as the heart of its filter kernel (where M is the data covariance matrix and d is the spectrum of the target of interest), then that algorithm is simply a CEM variant

67. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 67 Approved for Public Release 05-269 H º : p º (x)= J = # of Bands H 1 : p 1 (x)= Form the log-likelihood ratio test of H º and H 1 : Stocker, A.D., Reed, I.S., and Yu, X., (1990). Multi-dimensional signal processing for electro- Optical target detection. In: Signal and Data Processing of Small Targets 1990, Proceedings of the SPIE, v. 1305, pp. 218-231. Derivation taken from:

68. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 68 Approved for Public Release 05-269 Some algebra...

69. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 69 Approved for Public Release 05-269 A trick...recast as a univariable problem: After lots of simple algebra applied to the r.h.s: Now, go back to matrix-vector notation: a scalar threshold

70. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 70 Approved for Public Release 05-269 Take the natural log:...a scalar for each pixel { { Filter Kernel Pixel Threshold, T { >T for H 1 ; <T for H 0

71. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 71 Approved for Public Release 05-269 “The vector: Q T x is a projection of the original spectral data onto the eigenvectors of the covariance matrix, M, which corresponds to the principal axes of clutter distribution.” Stocker et al., 1990.

72. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 72 Approved for Public Release 05-269 “Further SCR gain is obtained by forming the optimum weighted combination of principal components using the weight vector:” From Stocker et al., 1990.

73. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 73 Approved for Public Release 05-269 Today’s HSI algorithms can also benefit from 1) spatial and spectral subsetting; 2) hierarchical application of techniques; 3) other... Its Important to Note That...

74. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 74 Approved for Public Release 05-269 What Should You Do? How Should You Do It? Join the fray!...HSI is a big tent Dump Lena/girl...  the spectrum is the fundamental datum Conduct honest, rigorous comparisons with existing, current best-practices HSI techniques Apply techniques to multiple, large, diverse data sets Team with HSI expert(s) and subject-matter expert(s) Seek peer-reviews from HSI experts  I’m happy to be a reviewer... Learn about/care about the field; be relevant  be more than buzz words  we’re more than an opportunity for statistical analyses

75. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 75 Approved for Public Release 05-269 Contact Information Ron Resmini The Boeing Company (Associate Technical Fellow); The National Geospatial-Intelligence Agency (NGA); and School of Computational Sciences, George Mason University v: 703-735-3899 e-mail(1): ronald.g.resmini@nga.mil e-mail(2): ronald.g.resmini@boeing.com

76. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 76 Approved for Public Release 05-269 Backup Slides

77. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 77 Approved for Public Release 05-269 Remote Sensing of Environment International Journal of Remote Sensing IEEE Transactions on Geoscience and Remote Sensing Journal of Geophysical Research Solid Earth, Planets, Oceans and Atmospheres Icarus Remote Sensing Reviews Photogrammetric Engineering and Remote Sensing Applied Optics Journal of the Optical Society of America Many others, too! Resources: Peer-Reviewed Journals

78. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 78 Approved for Public Release 05-269 Resources: Non-Reviewed Journals Published conference proceedings: SPIE your library may subscribe to SPIE abstract services AVIRIS (NASA) conference IEEE/IGARSS Proceedings of the ASPRS Many others, too!

79. NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY Know the Earth…Show the Way 79 Approved for Public Release 05-269 Linear Spectral Unmixing Theory Spectral unmixing theory states that the reflectance of an image pixel is a linear combination of reflectances from the (typically) several “pure” substances (or endmembers) contained within the ground-spot sampled by the remote sensing system. This is indicated below: where: R i is the reflectance of a pixel in band i, f j is the fractional abundance of substance (or endmember) j in the pixel, and M j,i is the reflectance of endmember substance j in band i. r i is the band-residual or unmodeled reflectance for the pixel in band i, and n is the number of endmembers. A spectral unmixing analysis results in n fraction-plane images showing the quantitative areal distribution of each of the endmember substances and one root mean squared (RMS) image showing an overall or global goodness of fit of the suite of endmembers for each pixel. The RMS image is formed, on a pixel-by-pixel basis, by: Objects may also be detected as anomalies in the RMS image.