Presentation is loading. Please wait.

Presentation is loading. Please wait.

Bormin Huang, Allen Huang, Alok Ahuja

Similar presentations


Presentation on theme: "Bormin Huang, Allen Huang, Alok Ahuja"— Presentation transcript:

1 Bormin Huang, Allen Huang, Alok Ahuja
Overview of NOAA/NESDIS GOES-R Hyperspectral Sounder Data Compression Study Bormin Huang, Allen Huang, Alok Ahuja Cooperative Institute for Meteorological Satellite Studies University of Wisconsin-Madison 4th MURI Workshop, April 27-28

2 What is hyperspectral sounding data?
It is generated from an interferometer (e.g. HIS, AERI, CrIS, IASI) or a grating sounder (e.g. AIRS). It consists of several thousand spectral channels that span the infrared region on the order of one wavenumber or less What is hyperspectral sounding data for? to retrieve - atmospheric temperature, water vapor, and trace gases profiles; - cloud & aerosol properties, - surface temperature, emissivities, etc., to derive wind from radiance or retrieved water vapor fields, for better weather and climate prediction. Why does it require compression? Unprecedented volume of 3D data that consists of one spectral and two spatial dimensions (~ GB per day) ; Beneficial to efficient data transfer and archive. What is new for the data compression society? High correlation among remote disjoint channels due to the absorption of the same absorbing gases. Lossy compression needs retrieval impact studies, i.e. interdisciplinary knowledge in data compression and remote sensing is needed! Why is lossless or near-lossless compression desired? Physical retrieval of atmospheric temperature and absorbing gases is a mathematically ill-posed problem, i.e. sensitive to data error and noise!

3 Lossless Compression Study
2D Wavelet-Based Compression Scheme JPEG2000: 2D IWT → Bitplane Coding → Entropy Coding 3D Wavelet-Based Compression Schemes 3D IWT → 3D EZW → Entropy Coding 3D IWT → 3D SPIHT → Entropy Coding 3D IWT → 1D BWT → Entropy Coding 2D Predictor-Based Compression Schemes CALIC : 2D Gradient-adjusted Prediction → Entropy Coding JPEG-LS : 2D Nonlinear Prediction → Entropy Coding

4 2D Wavelet Transform Integer Wavelet Transform (Lifting Scheme)

5 Wavelet based Schemes JPEG2000
A new ISO/IEC (International Organization for Standardization/International Electrotechnical Commission) compression standard. Successor to the DCT (discrete cosine transform)-based JPEG algorithm. IWT with 3 stages (Taubman et. al. 2000)

6 3D Wavelet Tree Coding 3D EZW: It uses the spatial hierarchical tree relationship of the wavelet transform coefficients for efficient compression. 3D SPIHT: Refinement of the EZW scheme that provides better compression while having faster encoding and decoding times. Parent-child interband relationship and locations for EZW and SPIHT coding

7 Predictor-Based Schemes
2D Context-based Adaptive Lossless Image Codec (CALIC) Among the nine proposals in the initial ISO/JPEG evaluation in July 1995, CALIC was ranked first. It is considered the benchmark for lossless compression of continuous-tone images. n ne nne nn nw w ww ? i j Neighboring pixels used in prediction (Wu et. al. 1997) Schematic description of the CALIC encoder

8 Published in 1999 as a lossless compression standard of the ISO/IEC.
2D JPEG-LS Published in 1999 as a lossless compression standard of the ISO/IEC. c b x d a Neighborhood of JPEG-LS used in prediction Schematic description of the JPEG-LS encoder

9 Burrows Wheeler Transform
Block-sorting compression scheme [Burrows et al, 1994] Rearranges the positions of the data such that the few distinct values under the same previous context are grouped together in position. tennessee*          tennessee* ennessee*t          *tennessee nnessee*te          ssee*tenne nessee*ten          e*tennesse essee*tenn          nnessee*te ssee*tenne          nessee*ten see*tennes          essee*tenn ee*tenness          see*tennes e*tennesse          ee*tenness *tennessee          ennessee*t             An example of the Burrows-Wheeler transform. bwt(tennessee*) = t*sennesee. The matrix on the right is obtained by sorting the rows of the left matrix in right-to-left lexicographic order. * denotes end of the data block and can be considered as the smallest symbol.

10 Ten selected AIRS granules on Sept. 6, 2002
00:53:31 UTC -12 H (Pacific Ocean, Daytime) Granule 16 01:35:31 UTC +2 H (Europe, Nighttime) Granule 60 05:59:31 UTC +7 H (Asia, Daytime) Granule 82 08:11:31 UTC -5 H (North America, Nighttime) Granule 120 11:59:31 UTC -10 H (Antarctica, Nighttime) Granule 126 12:35:31 UTC -0 H (Africa, Daytime) Granule 129 12:53:31 UTC -2 H (Arctic, Daytime) Granule 151 15:05:31 UTC +11 H (Australia, Nighttime) Granule 182 18:11:31 UTC +8 H (Asia, Nighttime) Granule 193 19:17:31 UTC -7 H (North America, Daytime) AIRS radiance field at wavenumber 900.3cm-1 for the selected granules

11 AIRS radiance field at wavenumber 900.3cm-1 for the selected granules

12 Compression ratios of different algorithms
for the 10 selected AIRS granules

13 Bias-Adjusted Reordering (BAR)* Scheme for Data Preprocessing
Hyperspectral sounder data features strong correlations in disjoint spectral regions affected by the same type of absorbing gases at various altitudes. The Bias-Adjusted Reordering (BAR) scheme is used for exploring the correlation among remote disjoint channels. The technique can be used to improve the compression ratio of any existing scheme. The BAR scheme paper is accepted to be published in Optical Engineering. We are in the process of patent application.

14 Effect of the BAR scheme on various compression algorithms for the 10 selected AIRS granules

15 Summary In support of the NOAA/NESDIS GOES-R data processing studies, we investigated lossless compression of 3D hyperspectral sounding data using wavelet-based schemes (3D EZW, 3D SPIHT, JPEG2000) and predictor-based schemes (CALIC, JPEG-LS). The performance rank from best to worst in terms of compression ratios before the BAR scheme is given in the order of JPEG-LS, 3D SPIHT, JPEG2000, CALIC, BWT and 3D EZW. The performance rank from best to worst in terms of compression ratios after the BAR scheme is given in the order of JPEG-LS, JPEG2000, CALIC, 3D SPIHT, BWT and 3D EZW. To take advantage of the spectral correlations, we applied the BAR scheme to significantly improve the compression performance of all the compression algorithms. Acknowledgement: This research is supported by NOAA NESDIS OSD under grant NA07EC0676.


Download ppt "Bormin Huang, Allen Huang, Alok Ahuja"

Similar presentations


Ads by Google