HDF 1 NCSA HDF XML Activities Robert E. McGrath Mike Folk National Center for Supercomputing Applications University of Illinois, Urbana-Champaign October 18, 2000

HDF 2 Summary This year we began to implement what is planned to be a complete set of tools for using XML and HDF5 to describe and manage scientific data.

HDF 3 Viewer/Editor Generator Dumper DDL Source code DDL ASCII XML HDF5 file XML SRB HDF5 file XML HDF5 file Input and output for HDF5 tools Java Non-Java DDL ASCII DIFF HDF5 and/or XML Diff file, etc. Done Doing Binary ? October, 2000 As of: ?

HDF 4 Work To Date The completed work includes: –A formal description of the HDF5 Data Model [1] –A design study of "Use Cases" for XML and HDF5 [2] –A Document Type Definition (DTD) for the HDF5 Data Model [3] –'h5gen', a Java tool that generates and HDF5 file from an XML description. [4]

HDF 5 Current Work Our current work includes: –Adding XML output to the 'h5dump' utility, to generate XML description of an HDF5 file –Adding XML ingest and output to the 'h5view' visual editor [4] –Revisions of the HDF5 DTD

HDF 6 Lessons Learned In this work, we have learned some important lessons. [5] –The "Use Cases" were especially instructive: it was clear that there are many possible uses of XML, with different requirements. –XML DTD's are unable to model numeric data effectively. –An HDF5 file may be structured as a rooted directed graph, this is difficult to represent in an XML tree structure.

HDF 7 XML Use Cases The HDF5 DTD will support a variety of uses: –Viewing structure and contents of HDF5 file using a Web browser –XML as a catalog record for locating datasets –XML as an intermediate form for programs –Generation, validation, and reconstruction of HDF5 files –XML as intermediate to other formal languages and file formats –Store XML in archive or in dataset as machine readable documentation –Templates, skeleton files, etc. See: [3]

HDF 8 Numeric Data E.g., an HDF5 dataset. 10 X 10, with 32-bit, signed Integers, big-endian. –XML can describe this datatype (see the HDF5 DTD) –but the contents of the array (100 numbers) can’t be expressed in any simple, standard way XML has no data types except “characters” Solution: Use XML Schema?

HDF 9 Expressing a Graph HDF5 file structure is not a tree, it is a rooted, directed graph. It may have ‘loops’, objects may have > 1 parent, >1 name, etc. XML enclosure maps nicely to a tree, but can’t express a graph or multiple parentage. Solution: several possibilities, see [5] Our approach: include the object in the tree once, use pointers for subsequent instances

HDF 10 Future Possibilities Additional future work may include: –Create an XML Schema for the HDF5 Data Model, this should be able to handle numbers effectively. –Explore the use of style sheets to transform between HDF5 and other data formats, such as netcdf.

HDF 11References 1. HDF5 Abstract Data Model, 2. Suggested Use Cases for XML with HDF-5, cases.html 3. HDF5-File.dtd, 4. HDF Java Tools, html 5. A DTD for HDF5: Design Notes,