Presentation on theme: "Bormin Huang, Allen Huang, Alok Ahuja"— Presentation transcript:
1 Bormin Huang, Allen Huang, Alok Ahuja Overview of NOAA/NESDIS GOES-R Hyperspectral Sounder Data Compression StudyBormin Huang, Allen Huang, Alok AhujaCooperative Institute for Meteorological Satellite StudiesUniversity of Wisconsin-Madison4th MURI Workshop, April 27-28
2 What is hyperspectral sounding data? It is generated from an interferometer (e.g. HIS, AERI, CrIS,IASI) or a grating sounder (e.g. AIRS).It consists of several thousand spectral channels that span theinfrared region on the order of one wavenumber or lessWhat is hyperspectral sounding data for?to retrieve- atmospheric temperature, water vapor, and trace gases profiles;- cloud & aerosol properties,- surface temperature, emissivities, etc.,to derive wind from radiance or retrieved water vapor fields,for better weather and climate prediction.Why does it require compression?Unprecedented volume of 3D data that consists of one spectraland two spatial dimensions (~ GB per day) ;Beneficial to efficient data transfer and archive.What is new for the data compression society?High correlation among remote disjoint channels due to theabsorption of the same absorbing gases.Lossy compression needs retrieval impact studies, i.e.interdisciplinary knowledge in data compression andremote sensing is needed!Why is lossless or near-lossless compression desired?Physical retrieval of atmospheric temperature andabsorbing gases is a mathematically ill-posed problem,i.e. sensitive to data error and noise!
5 Wavelet based Schemes JPEG2000 A new ISO/IEC (International Organization for Standardization/International Electrotechnical Commission) compression standard.Successor to the DCT (discrete cosine transform)-based JPEG algorithm.IWT with 3 stages (Taubman et. al. 2000)
6 3D Wavelet Tree Coding3D EZW: It uses the spatial hierarchical tree relationship of the wavelet transform coefficients for efficient compression.3D SPIHT: Refinement of the EZW scheme that provides better compression while having faster encoding and decoding times.Parent-child interband relationship and locations for EZW and SPIHT coding
7 Predictor-Based Schemes 2D Context-based Adaptive Lossless Image Codec (CALIC)Among the nine proposals in the initial ISO/JPEG evaluation in July 1995, CALIC was ranked first.It is considered the benchmark for lossless compression of continuous-tone images.nnennennnwwww?ijNeighboring pixels used in prediction (Wu et. al. 1997)Schematic description of the CALIC encoder
8 Published in 1999 as a lossless compression standard of the ISO/IEC. 2D JPEG-LSPublished in 1999 as a lossless compression standard of the ISO/IEC.cbxdaNeighborhood of JPEG-LSused in predictionSchematic description of the JPEG-LS encoder
9 Burrows Wheeler Transform Block-sorting compression scheme [Burrows et al, 1994]Rearranges the positions of the data such that the few distinct values under the same previous context are grouped together in position.tennessee* tennessee*ennessee*t *tennesseennessee*te ssee*tennenessee*ten e*tennesseessee*tenn nnessee*tessee*tenne nessee*tensee*tennes essee*tennee*tenness see*tennese*tennesse ee*tenness*tennessee ennessee*tAn example of the Burrows-Wheeler transform.bwt(tennessee*) = t*sennesee. The matrix on the right is obtained by sorting the rows of the left matrix in right-to-left lexicographic order. * denotes end of the data block and can be considered as the smallest symbol.
11 AIRS radiance field at wavenumber 900.3cm-1 for the selected granules
12 Compression ratios of different algorithms for the 10 selected AIRS granules
13 Bias-Adjusted Reordering (BAR)* Scheme for Data Preprocessing Hyperspectral sounder data features strong correlations in disjoint spectral regions affected by the same type of absorbing gases at various altitudes.The Bias-Adjusted Reordering (BAR) scheme is used for exploring the correlation among remote disjoint channels.The technique can be used to improve the compression ratio of any existing scheme.The BAR scheme paper is accepted to be published in Optical Engineering.We are in the process of patent application.
14 Effect of the BAR scheme on various compression algorithms for the 10 selected AIRS granules
15 SummaryIn support of the NOAA/NESDIS GOES-R data processing studies, we investigated lossless compression of 3D hyperspectral sounding data using wavelet-based schemes (3D EZW, 3D SPIHT, JPEG2000) and predictor-based schemes (CALIC, JPEG-LS).The performance rank from best to worst in terms of compression ratios before the BAR scheme is given in the order of JPEG-LS, 3D SPIHT, JPEG2000, CALIC, BWT and 3D EZW.The performance rank from best to worst in terms of compression ratios after the BAR scheme is given in the order of JPEG-LS, JPEG2000, CALIC, 3D SPIHT, BWT and 3D EZW.To take advantage of the spectral correlations, we applied the BAR scheme to significantly improve the compression performance of all the compression algorithms.Acknowledgement: This research is supported by NOAA NESDIS OSD under grant NA07EC0676.