Spatial and Spectral Evaluation of Image Fusion Methods Sascha Klonus Manfred Ehlers Institute for Geoinformatics and Remote Sensing University of Osnabrück

Content Introduction Image Fusion Test Site Fusion Results Color Distortions Evaluation Methods and Results Ehlers Fusion Conclusions and Future Work

Remote sensors have different spatial resolution for panchromatic and multispectral imagery The ratios vary between 1:2 and 1:5 For multisensor fusion the ratios can exceed 1:30 (e.g. Ikonos/Landsat) Data Fusion: Why is it Necessary?

Objectives of Image Fusion Sharpen images Improve geometric corrections Provide stereo-viewing capabilities Enhance certain features Complement data sets Detect changes Substitute missing information Replace defective data Pohl & van Genderen (1998)

Meaning of Pan-Sharpening SpatialSpectral + panchromatic & high geometric resolution multi-/hyperspectral image & low geometric resolution multi-/hyperspectral & high geometric resolution

Fusion Methods Color Transformations  Modified IHS Transformation Statistical Methods  Principal Component Merge Numerical Methods  Brovey  CN Spectral Sharpening  Gram-Schmidt Spectral Sharpening  Wavelet based Fusion Combined Methods  Ehlers Fusion

Test Site

Original Data Quickbird Multispectral image ( ) Quickbird Panchromatic image ( ) Formosat Multispectral image ( ) Ikonos Multispectral image ( )

Single Sensor Fusion: Quickbird Quickbird Multispectral image Fused with BroveyFused with CN Spectral SharpeningFused with Ehlers Fused with Wavelet Fused with Gram-Schmidt Fused with PC Fused with modified IHS

Multisensor Fusion: Ikonos Ikonos Multispectral image Fused with BroveyFused with CN Spectral Sharpening Fused with Ehlers Fused with modified IHS Fused with PCFused with Gram-Schmidt Fused with Wavelet

Multisensor Fusion: Formosat Formosat Multispectral image Fused with Brovey Fused with CN Spectral Sharpening Fused with Ehlers Fused with modified IHSFused with PCFused with Gram-Schmidt Fused with Wavelet

Panchromatic band has a different spectral sensitivity Multisensoral differences (e.g. Ikonos and SPOT merge) Multitemporal (seasonal) changes between pan and ms image data Fusion Problem: Color Distortion  Inconsistent panchromatic information is fused into the multispectral bands

Spectral Comparison Methods (1) s = standard deviation org = Original image fused = Fused image x = Mean RMSE Correlation coefficients Visual (Structure and Colour Preservation)

Results RMSE QuickbirdIkonosFormosat Mod. IHS PC Brovey CN-Sharpening Gram-Schmidt Wavelet Ehlers

Results Correlation Coefficients QuickbirdIkonosFormosat Mod. IHS PC Brovey CN-Sharpening Gram-Schmidt Wavelet Ehlers

Spectral Comparison Methods (2) Per Pixel Deviation Degrade Degraded to ground resolution of original image (Formosat = 8m) Original multispectral image (Formosat 8m) Band Band Band Band Result: Vector containing the deviation per pixel Fused image (Formosat 2m)

Mean Per Pixel Deviation QuickbirdIkonosFormosat IHS PC Brovey CN-Sharpening Gram-Schmidt Wavelet Ehlers

Spatial Comparison Methods (1) Edge Detection Band % Band % Band % Mean91.96 %

Results Edge Detection QuickbirdIkonosFormosat Mod. IHS92.71 %92.44 %95.54 % PC95.10 %93.28 %93.44 % Brovey94.69 %95.16 %97.87 % CN-Sharpening94.69 %95.16 %90.69 % Gram-Schmidt95.02 %95.53 %97.82 % Wavelet85.00 %83.82 %84.81 % Ehlers91.85 %90.35 %94.40 %

Spatial Comparison Methods (2) Highpass Filtering Correlation Band Band Band Mean0.7918

Highpass Correlation Results QuickbirdIkonosFormosat Mod. IHS PC Brovey CN-Sharpening Gram-Schmidt Wavelet Ehlers

FFT Filter Based Data Fusion (Ehlers Fusion) Panchromatic Image Multispectral Image RGBRGB Basis: IHS Transform and Filtering in the Fourier Domain FFT Fourier Spectrum FFT Fourier Spectrum HPF Pan HP LPF I LP IHSIHS R‘ G‘ B‘ IHS -1 I LP +Pan HP H S FFT -1

Panchromatic image and its spectrum Original panchromatic image Panchromatic Spectrum

Filtersetting effects Frequency Intensity Cut-off Frequency fnfn Filtered Panchromatic Spectrum

Effects in the spatial domain Filtered panchromatic imageFused image

Filtersetting effects Frequency Intensity Cut-off Frequency fnfn Filtered Panchromatic Spectrum

Effects in the spatial domain Filtered panchromatic imageFused image

Filtersetting effects Filtered Panchromatic Spectrum Frequency Intensity Cut-off Frequency fnfn

Effects in the spatial domain Filtered panchromatic imageFused image

Results Ehlers Fusion shows the best overall results in all images It works also if the panchromatic Information does not match the spectral sensitivity of the merged bands (multitemporal and multisensoral fusion) Its performance is superior to standard fusion techniques (IHS, Brovey Transform, PC Merge) Wavelet preserves the spectral characteristics at the cost of spatial improvement Ehlers Fusion is integrated in a commercial image processing system (Erdas Imagine 9.1)

Future Work Fusion of radar- and optical Data Development of one method to evaluate the spatial and spectral quality of an fused image Comparison with the algorithm of Zhang (PCI Geomatica) Research on automation for filter design

Thanks for your Attention Questions???

Ehlers Fusion Program

Multispectral image and its spectrum Original multispectral intensity Multispectral intensity spectrum

Filtersetting effects Filtered multispectral spectrum Frequency Intensity Cut-off Frequency fnfn

Filtersetting effects Filtered multispectral spectrum Frequency Intensity Cut-off Frequency fnfn

Filtersetting effects Filtered multispectral spectrum Frequency Intensity Cut-off Frequency fnfn