1. SAN DIEGO SUPERCOMPUTER CENTER HDF5/SRB Integration June 14, 2006 Mike Wan mwan@sdsc.edu mwan@sdsc.edu SRB, SDSC Peter Cao xcao@ncsa.edu xcao@ncsa.edu HDF, NCSA Sponsored by CIP/NLADR, NFS PACI Project in Support of NCSA-SDSC Collaboration

2. SAN DIEGO SUPERCOMPUTER CENTER Current Status Publish HDF5 AIP documents White paper: http://hdf.ncsa.uiuc.edu/hdf-aip-html/http://hdf.ncsa.uiuc.edu/hdf-aip-html/ HDF5 METS template: http://hdf.ncsa.uiuc.edu/hdf-aip-html/hdf5_mets_template.xml http://hdf.ncsa.uiuc.edu/hdf-aip-html/hdf5_mets_template.xml Finish h5ingest command line tool Create HDF5 METS template file Validate HDF5 METS document Work on test suite and bug fix (Peter Cao) Work on performance improvement (Mike Wan) Setup a demo server to support SCEC files

3. SAN DIEGO SUPERCOMPUTER CENTER Potential SAC Projects SDSC ENZO project Enzo, 3D cosmological hydrodynamics code, simulating the process of massive star formation and destruction HDF5 is used as file format and parallel file I/O access FLASH Program The UC/DOE collaboration on creating three-dimensional, virtual reality projections of the cosmic explosions HDF5 is used for storing the data and high I/O access SCEC Terascale Earthquake Simulations Over 100 TB data/year Collections at SRB – 2.6 million files, 114 Terabytes

4. SAN DIEGO SUPERCOMPUTER CENTER TeraShake Surface Seismograms 4D Array (1.2 TB) Time (22,728) Horizontal (3,000) Vertical (1,500) Vector Component (3) Each file: 22,728 x 3,000 x 5 x 1 1,363,680,000 Bytes TeraShake scenario 900 files

5. SAN DIEGO SUPERCOMPUTER CENTER Example HDF5 File xhist00001hpss-scec13636800002005-02-23-22.37 xhist00002hpss-scec13636800002005-02-23-22.38 xhist00003hpss-scec13636800002005-02-23-22.39 xhist00004hpss-scec13636800002005-02-23-22.40 xhist00005hpss-scec13636800002005-02-23-22.40 HDF5 File 32-bit float 22,728 3,000 25

6. SAN DIEGO SUPERCOMPUTER CENTER File on SRB server

7. SAN DIEGO SUPERCOMPUTER CENTER Select a Subset

8. SAN DIEGO SUPERCOMPUTER CENTER HDFView