1. www.hdfgroup.org The HDF Group HDF/HDF-EOS Workshop XIV1 Easy Access of NASA HDF data via OPeNDAP Kent Yang and Joe Lee The HDF Group September 28,2010

2. www.hdfgroup.org Easy HDF/HDF-EOS Workshop XIV2September 28,2010

3. www.hdfgroup.org Outline Introduction to OPeNDAP HDF4/5-OPeNDAP Handlers Live Demo HDF/HDF-EOS Workshop XIV3September 28,2010

4. www.hdfgroup.org OPeNDAP A software framework that allows simple access to remote data Client-server model Servers make the local data accessible remotely Clients make applications easily access to remote served data Widely used by Earth Science Community URL: http://www.opendap.org/ September 28,2010HDF/HDF-EOS Workshop XIV4

5. www.hdfgroup.org Access HDF via FTP HDF/HDF-EOS Workshop XIV5 Users FTP Clients IDL FTP Servers HDF Files AIRS/OMI FTP Local HDF Files Aqua/Aura September 28,2010

6. www.hdfgroup.org Client Library (libnc_dap) Data Access Protocol (DAP) via Internet Server (Hyrax) Handler (hdf4/5_handler) Remote Data (HDF4/5) View Data (User) OPeNDAP Concept 6 Visualization Tools (gradsdap) Typical Remote Access FTP/HTTP Visualization tools (e.g., grads) HDF/HDF-EOS Workshop XIVSeptember 28,2010

7. www.hdfgroup.org Access HDF via OPeNDAP HDF/HDF-EOS Workshop XIV7 Users OPeNDAP Clients IDV OPeNDAP Servers HDF Files Hyrax AIRS/OMI DAP libnc-dap Visualization Tools HDF HANDLER Aqua/Aura September 28,2010

8. www.hdfgroup.org OPeNDAP Data Types OPeNDAP provides three basic data objects: -data descriptor structure (.dds) syntactic metadata – rigid -data attribute structure (.das) semantic metadata – flexible -data (.dods) - the actual data in a binary structure Data are requested with a URL September 28,2010HDF/HDF-EOS Workshop XIV8

9. www.hdfgroup.org OPeNDAP Servers September 28,2010HDF/HDF-EOS Workshop XIV9 Data Matlab HDF4/5 JDBC FreeFromFITS CDFCEDAR Data netCDF HDF4/5 Matlab Data DSP Data JGOFS TablesSQLFITSCDF Flat Binary CEDAR Data CODAR Data ESML GeneralCODAR

10. www.hdfgroup.org OPeNDAP Clients HDF/HDF-EOS Workshop XIV10 netCDF C netCDF Java IDVFerretGrADSVisADncBrowse MatlabExcelIDLAccess Matlab Client IDL Client September 28,2010

11. www.hdfgroup.org Outline Introduction to OPeNDAP HDF4/5-OPeNDAP Handlers Live Demo HDF/HDF-EOS Workshop XIV11September 28,2010

12. www.hdfgroup.org Enhanced HDF4 Handler HDF/HDF-EOS Workshop XIV12 Users OPeNDAP Clients IDV OPeNDAP Servers HDF4 Files Hyrax DAP libnc-dap Aura MODIS Swath w/ Dim. Map Enhanced HDF4 HANDLER Visualization Tools September 28,2010

13. www.hdfgroup.org An example: Access HDF data via OPeNDAP HDF/HDF-EOS Workshop XIV13 Swath with Dimension Map September 28,2010

14. www.hdfgroup.org HDF-EOS2 Products Supported GES DISC AIRS / MERRA / TOMS LAADS / LP DAAC / NSIDC Many MODIS products NSIDC AMSR_E / NISE LaRC MISR / MOPITT / some CERES(TRMM) HDF/HDF-EOS Workshop XIV14September 28,2010

15. www.hdfgroup.org Pure HDF4 Products Supported GES DISC TRMM L1B, L2B Swath TRMM L3 Grid 42B & 43B products OBPG (Ocean Color) SeaWiFS / MODIST / MODISA CZCS / OCTS HDF/HDF-EOS Workshop XIV15September 28,2010

16. www.hdfgroup.org Pure HDF4 Products Supported CERES CER_AVG CER_ES4 CER_ISCCP-D2like-Day CER_ISCCP-D2like-GEO CER_SRBAVG CER_SYN CER_ZAVG HDF/HDF-EOS Workshop XIV16September 28,2010

17. www.hdfgroup.org Limitations Reasons for Visualization Failures IDV / Panoply / Java Heap Memory Network / DAP (e.g., max HTTP header size) Can’t retrieve key information correctly (e.g., LAMAZ projection) from data Unable to harmonize with CF-conventions (e.g., FillValue inside latitude and longitude) HDF/HDF-EOS Workshop XIV17September 28,2010

18. www.hdfgroup.org Limitations(continued) HDF-EOS2 + additional HDF4 objects Additional HDF4 objects will be ignored. The rest of pure HDF4 products not listed in previous slides are not tested and may not be visualized by Java OPeNDAP Clients. HDF/HDF-EOS Workshop XIV18September 28,2010

19. www.hdfgroup.org HDF-EOS5 products Supported OMI Swath, Grid MLS/HIRDLS Swath TES Grid September 28,2010HDF/HDF-EOS Workshop XIV19

20. www.hdfgroup.org Learning Curve of accessing HDF data September 28,2010HDF/HDF-EOS Workshop XIV20 From the ESIP wiki page: http://wiki.esipfed.org/index.php/Making_Science_Data_Easier_to_Use_with_OPeNDAP Making Science Data Easier to Use with OPeNDAP NCL

21. www.hdfgroup.org Live Demo HDF/HDF-EOS Workshop XIV21September 28,2010

22. www.hdfgroup.org OPeNDAP HDF4/HDF5 handlers HDF/HDF-EOS Workshop XIV22September 28,2010 Source codes and RPMs can be found under http://opendap.org Demo HDF5/HDF4 servers http://hdfeos.org/zoo/hdf5_handler/ Currently NASA GES DISC has installed both OPeNDAP HDF4 and HDF5 handlers

23. www.hdfgroup.org The HDF Group HDF/HDF-EOS Workshop XIV23 Thank you ! September 28,2010

24. www.hdfgroup.org Acknowledgements This work was supported by cooperative agreement number NNX08AO77A from the National Aeronautics and Space Administration (NASA). Any opinions, findings, conclusions, or recommendations expressed in this material are those of the author[s] and do not necessarily reflect the views of the National Aeronautics and Space Administration. HDF/HDF-EOS Workshop XIV24September 28,2010

25. www.hdfgroup.org The HDF Group Questions/comments? HDF/HDF-EOS Workshop XIV25September 28,2010

26. www.hdfgroup.org Our Work Let Visualization Clients Access HDF- EOS2 and HDF4 files HDF/HDF-EOS Workshop XIV26September 28,2010

27. www.hdfgroup.org HDF5-OPeNDAP Handler Update Support Aura HDF-EOS5 OMI Swath Access some files that follow HDF5 Dimension Scale Specification Support Aura HDF-EOS5 MLS HDF/HDF-EOS Workshop XIV27September 28,2010