Important ESDIS 2009 tasks review Kent Yang, Mike Folk The HDF Group April 1st, /1/20151Annual briefing to ESDIS

Outline HDF-EOS Website HDF OPeNDAP support HDF-EOS5 Augmentation tool 10/1/20152Annual briefing to ESDIS

HDF-EOS Web Site 10/1/20153Annual briefing to ESDIS

Main page : Reorganize the overall structure Update the main page 10/1/ Annual briefing to ESDIS

Help: 10/1/ Annual briefing to ESDIS

Examples Added a new “Examples” section - C, Fortran, IDL and MatLab examples to access HDF-EOS2 and HDF-EOS5 Grid and Swath files - Examples on how to use tools such as HEG, GrADS, NCL, Using IDV via OPeNDAP to access HDF-EOS 2 and HDF-EOS5 data 6 10/1/20156Annual briefing to ESDIS

Software Evaluated 35 tools and libraries Added 11 tools and libraries Added detailed descriptions November 3-5, 20097HDF/HDF-EOS Workshop XIII 10/1/20157Annual briefing to ESDIS

November 3-5, 2009HDF/HDF-EOS Workshop XIII8 Forum 10/1/20158Annual briefing to ESDIS

HDF-EOS website statistics Current Period: Nov. 1 st 2009 – Feb. 28 th 2010 Previous Period: July 4 th 2009 – Oct. 31 st /1/20159Annual briefing to ESDIS

Definitions Visits – The total number of visits to the website. Pageviews – The total number of pages viewed on the website. Repeated views of a single page are counted. Pages/Visit – The average number of page views per visitor. Bounce Rate – The percentage of single-page visits. (Total number of visits viewing one page) / (Total number of visits) Avg. Time on Site – The average visit duration. % New visits – The percentage of visits by people who had never visited the website before. Organic – Web traffic which comes from unpaid listing at search engines or directories. 10/1/201510Annual briefing to ESDIS

HDF-EOS Website Statistics Comparison JulAugSep Oct 10/1/201511Annual briefing to ESDIS

Month by Month Visits Comparison 10/1/201512Annual briefing to ESDIS

Traffic Sources - Visitors who typed in the URL directly -Visitors who clicked a link from another website - Visitors who used a search engine 10/1/201513Annual briefing to ESDIS

10/1/201514Annual briefing to ESDIS

Pageviews 10/1/201515Annual briefing to ESDIS

Top Pages Viewed 10/1/201516Annual briefing to ESDIS

10/1/201517Annual briefing to ESDIS

10/1/201518Annual briefing to ESDIS

10/1/201519Annual briefing to ESDIS

10/1/201520Annual briefing to ESDIS

Current work 10/1/201521Annual briefing to ESDIS

Current Work Collect sample files for NASA HDF4/HDF-EOS2 products Provide special HDF-EOS description for each sample file Examine the access of each sample file with widely used tools Provide the codes and the plots generated from the tools for each sample file 10/1/201522Annual briefing to ESDIS

Tools to access the HDF data NCL MatLab IDL 10/1/201523Annual briefing to ESDIS

10/1/ Datapool Example 10/1/201524Annual briefing to ESDIS

HDF OPeNDAP Work Update 10/1/201525Annual briefing to ESDIS

HDF5 –OPeNDAP Handler Four official releases from OPeNDAP website The latest release date is February 2010 Add the support for HDF-EOS5 Aura swath files Aura MLS, OMI, HIRDLS files are served via OPeNDAP URL: /1/201526Annual briefing to ESDIS

Access Aura MLS data via OPeNDAP 10/1/201527Annual briefing to ESDIS

HDF4 OPeNDAP Handler 10/1/201528Annual briefing to ESDIS

Phase 1 work review Time: April 2009 – October 2009 Enhanced the original HDF4-OPeNDAP handler Some HDF-EOS2 data can be served via OPeNDAP Provided a version to GES DISC 10/1/201529Annual briefing to ESDIS

Phase 2 work Time: November 2009 – July 2010 Goal: Enhance the HDF4-OPeNDAP handler so that more NASA HDF-EOS2 and HDF4 data can be served via OPeNDAP 10/1/201530Annual briefing to ESDIS

Phase 2 work Comprehensive examinations of all NASA HDF- EOS2 data products - NASA Data pool in the previous slides Come up with the re-design based on the examinations Re-implement the HDF-EOS2 module of HDF4- OPeNDAP handler Add the support for handling some NASA pure HDF4 data 10/1/201531Annual briefing to ESDIS

Phase 2 - Current Status We finished the examination of NASA HDF-EOS2 products We finished the design note of HDF-EOS2 module of HDF4-OPeNDAP handler We finished implementing 90% features of the HDF-EOS2 module We provided a customized version for GES DISC to test 10/1/201532Annual briefing to ESDIS

Phase 2 – Current Work Highlights Focus on making the data accessible by Java- based OPeNDAP clients Strictly Follow CF/COARD conventions Handle different HDF-EOS2 cases 10/1/201533Annual briefing to ESDIS

Phase 2 work plan Time: Now to July 14 th, 2010 Finish all features for the HDF-EOS2 module Handle Hybrid HDF-EOS2 files Handle some NASA HDF4 files 10/1/201534Annual briefing to ESDIS

HDF-EOS5 augmentation tool 10/1/201535Annual briefing to ESDIS

HDF-EOS5 and NetCDF-4 Enabling NetCDF4 to access HDF-EOS5 data  Conversion  Augmentation One file can be used for both EOS5 and NetCDF-4. Note that EOS5 users are not affected at all. HDF5 HDF-EOS5 file HDF5 NetCDF4 NetCDF-4 file Augmented HDF-EOS5 file Augmentation 10/1/201536Annual briefing to ESDIS

Goals Augmented file must be readable through  NetCDF-4 API as well as HDF-EOS5 API How?  Read info in the HDF-EOS5 part.  Transform it to conform to NetCDF-4 model..  Append the new info. + = 10/1/201537Annual briefing to ESDIS

Past Work Review GridSwath OMI L2G OMI L3G TES MLS HIRDLS OMI MLS HIRDLS TES Not all Aura HDF-EOS5 files use projection for storing dimension values. For example, OMI L2G grid files store lat/lon in data fields. TES grid files use both ways. Collected and Examined different HDF-EOS5 data 10/1/201538Annual briefing to ESDIS

Past Work Review To follow the netCDF4 data model, we need users to provide the information about the dimension fields. MLS and HIRDLS data are vertical profilers. The dimension fields are Pressure and Time. We need similar information for other instruments. Provided a customized version for HIRDLS team 10/1/201539Annual briefing to ESDIS

Future Work Handle other cases: OMI L2G,future HIRDLS/MLS grid Design a general input file for insufficient information Continue testing with NetCDF-4 snapshots Consistency checker  If one modifies dim info in an augmented file, file becomes inconsistent. Testing, Documentation 10/1/201540Annual briefing to ESDIS

Thank You! 10/1/201541Annual briefing to ESDIS

Acknowledgements This work was supported by the Cooperative Agreement with the National Aeronautics and Space Administration (NASA) under NASA grant NNX06AC83A and NNX08A077A. Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of NASA. 10/1/201542Annual briefing to ESDIS

Questions/comments? 10/1/201543Annual briefing to ESDIS

Backup slides 10/1/201544Annual briefing to ESDIS

Role of HDF4-OPeNDAP handler OPeNDAP ClientsOPeNDAPHDF-EOS2 HDF4-OPeNDAP handler 10/1/201545Annual briefing to ESDIS