1. SAN DIEGO SUPERCOMPUTER CENTER HDF5/SRB Integration August 28, 2006 Mike Wan mwan@sdsc.edu mwan@sdsc.edu SRB, SDSC Peter Cao xcao@ncsa.edu xcao@ncsa.edu HDF, NCSA Sponsored by CIP/NLADR, NFS PACI Project in Support of NCSA-SDSC Collaboration

2. SAN DIEGO SUPERCOMPUTER CENTER Current Status Improved transferring raw data (SDSC) Tested performance test #1 (270MB) from 18 to 15 (seconds) test #2 (550MB) from 231 to 35 (seconds) test #3 (820MB) from 734 to 113 (seconds) Project plan for SRB-FLASH

3. SAN DIEGO SUPERCOMPUTER CENTER Things to move forward Improve GUI component in HDFView to open remote files (NCSA) Support primitive writing capabilities for HDF5 files in SRB (NCSA and SDSC) Write documentation and extensive testing (NCSA and SDSC) Release the production system and publish documentation (NCSA and SDSC)

4. SAN DIEGO SUPERCOMPUTER CENTER SAP/SAC program: FLASH Slice (gas temperature) and isosurface (gas density) in particle/mesh simulation FLASH is a adaptive-mesh simulation code for astrophysical hydrodynamicsFLASH FLASH slice extractor retrieves a specific 2D slice from a 3D AMR (adaptive mesh refinement)

5. SAN DIEGO SUPERCOMPUTER CENTER SRB-FLASH: architecture

6. SAN DIEGO SUPERCOMPUTER CENTER SRB-FLASH: benefits No need for large storage on local machine. Terabytes of data files sit only on the sever machine Instant access to the data and metadata. Since only the selected data/metadata are transferred to the client instead of the whole multi-terabyte file, time on the network is tremendously reduced Data sharing among scientists. Since data sits on the server machine, it can be shared with scientists without multiple local copies of the same file. Also, everyone will have access to the updated data after a new simulation run

7. SAN DIEGO SUPERCOMPUTER CENTER SRB-FLASH: project plan This project will be divided into three phases: phase I, phase II and phase III Phase I and part of phase II are required Phase III and part of phase II are optional, depending on the duration and amount of the funding The time estimates are based on an half (50%) FTE

8. SAN DIEGO SUPERCOMPUTER CENTER SRB-FLASH: milestone Phase I: a command line client utility and server function to support selection of FLASH AMR mesh data. TasksStartFinish Collaboration between NCSA and SDSCDuration of the project Collecting requirements10/2/0610/13/06 Exploying the current tool10/16/0610/27/06 Implementing server function10/30/0612/22/06 Implementing client utility12/25/062/16/07 Testing with FLASH AMR mesh data2/19/073/16/07 Deploring to evaluation server3/19/074/13/07 Documentation and release4/16/075/11/07

9. SAN DIEGO SUPERCOMPUTER CENTER SRB-FLASH: milestone Phase II: enhanced utilities querying FLASH specific metadata and statistics, and performing data interpolation and simple mathematic manipulation. TasksStartFinish Collaboration between NCSA and SDSCDuration of the project Query and return simple statistics5/14/076/22/07 Extract and return particle information6/25/078/17/07 Analyze and interpolate line segment8/20/0710/12/07 Add support for png and ASCII text10/15/0711/23/07

10. SAN DIEGO SUPERCOMPUTER CENTER SRB-FLASH: milestone Phase III: development of general framework that can be adapted to other applications. We need to scope out the milestone for this phase based on the experience of phase I and II.