We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byKelsi Redburn
Modified over 4 years ago
www.hdfgroup.org The HDF Group Parallel HDF5 Developments 1 Copyright © 2010 The HDF Group. All Rights Reserved Quincey Koziol The HDF Group firstname.lastname@example.org
www.hdfgroup.org 2 Goal is to be invisible: get same performance with HDF5 as with MPI I/O Project with LBNL/NERSC to improve HDF5 performance on parallel applications: 6-12x performance improvements on various applications (so far) Parallel I/O in HDF5 Copyright © 2010 The HDF Group. All Rights Reserved
www.hdfgroup.org 3 Up to 12GB/s to shared file (out of 15GB/s) on NERSCs franklin system: Parallel I/O In HDF5 Copyright © 2010 The HDF Group. All Rights Reserved
www.hdfgroup.org The HDF Group Recent Improvements to Parallel HDF5 4 Copyright © 2010 The HDF Group. All Rights Reserved
www.hdfgroup.org 5 Reduce number of file truncation operations Distribute metadata I/O over all processes Detect same shape of selection in more cases, allowing optimized I/O path to be taken more often Many other, smaller, improvements to library algorithms for faster/better use of MPI Recent Parallel I/O Improvements Copyright © 2010 The HDF Group. All Rights Reserved
www.hdfgroup.org Reduced File Truncations HDF5 library was very conservative about truncating file when H5Fflush called. However, file truncation very expensive in parallel. Library modified to defer truncation until file closed. Copyright © 2010 The HDF Group. All Rights Reserved6
www.hdfgroup.org Distributed Metadata Writes HDF5 caches metadata internally, to improve both read and write performance Historically, process 0 writes all dirtied metadata to HDF5 file, while other processes wait Changed to distribute ranges of metadata within the file across all processes Results in ~10x improvement in I/O for Vorpal (see next slide) Copyright © 2010 The HDF Group. All Rights Reserved7
www.hdfgroup.org Dsitributed Metadata Writes I/O Trace Before Changes Note long sequence of I/O from process 0 I/O Trace After Changes Note distribution of I/O across all processes, taking much less time Copyright © 2010 The HDF Group. All Rights Reserved8
www.hdfgroup.org Improved Selection Matching When HDF5 performs I/O between regions in memory and the file, it compares the regions to see if the applications buffer can be directly used for I/O Historically, this algorithm couldnt detect that a region with the same shape, but embedded in arrays of different dimensionality were the same For example, a 10x10 region in a 2-D array should compare equal to the equivalent 1x10x10 region in a 3-D array Changed to detect same shaped region in arbitrary source and destination buffer array dimensions, allowing I/O from applications buffer in more circumstances. Copyright © 2010 The HDF Group. All Rights Reserved9
www.hdfgroup.org Improved Selection Matching Change resulted in ~20x I/O performance improvement when reading 1-D buffer from 2-D file dataset From ~5-7 seconds (or worse) to ~0.25-0.5 seconds, on a variety of machine architectures (Linux: amani, hdfdap, jam; Solaris: linew) Copyright © 2010 The HDF Group. All Rights Reserved10
www.hdfgroup.org The HDF Group Upcoming Improvements to Parallel HDF5 11 Copyright © 2010 The HDF Group. All Rights Reserved
www.hdfgroup.org 12 HPC environments typically have unusual, possibly even unique, computing, network and storage configurations. The HDF5 distribution should provide easy to use interfaces that ease scientists and developers use of these platforms: Tune and adapt to the underlying parallel file system. New high-level API routines that wrap existing HDF5 functionality in a way that is easier for HPC application developers to use and help them move applications from one HPC environment to another. RFC: http://www.hdfgroup.uiuc.edu/RFC/HDF5/HPC-High-Level- API/H5HPC_RFC-2010-09-28.pdfhttp://www.hdfgroup.uiuc.edu/RFC/HDF5/HPC-High-Level- API/H5HPC_RFC-2010-09-28.pdf High-Level HPC API for HDF5 Copyright © 2010 The HDF Group. All Rights Reserved
www.hdfgroup.org 13 File System Tuning: Automatic file system tuning Pass file system tuning info to HDF5 library Convenience Routines: Macro routines Encapsulate common parallel I/O operations E.g. - create a dataset and write a different hyperslab from each process, etc. Extended routines Provide special parallel I/O operations not available in main HDF5 API Examples: Group collective I/O operations Collective raw data I/O on multiple datasets Collective multiple object manipulation Optimized collective object operations High-Level HPC API for HDF5 – API Overview Copyright © 2010 The HDF Group. All Rights Reserved
www.hdfgroup.org The HDF Group Parallel HDF5 in the Future 14 Copyright © 2010 The HDF Group. All Rights Reserved
www.hdfgroup.org 15 DOE Exascale FOA w/LBNL & PNNL Proposal Funded Exascale-focused enhancements to HDF5 LLNL Support & Development Contract Performance, support and medium-term focused development DOE Exascale FOA w/ANL and ORNL Proposal Funded Research on alternate file formats for Exascale I/O LBNL Development Contract Performance and short-term focus HPC Funding in 2010 and Beyond Copyright © 2010 The HDF Group. All Rights Reserved
www.hdfgroup.org 16 Library Enhancements Proposed: Remove collective metadata modification restriction Append-only mode, targeting restart files Embarrassingly parallel mode, for decoupled applications Overlapping compute & I/O, with asynchronous I/O Auto-tuning to underlying parallel file system Improve resiliency of changes to HDF5 files Bring FastBit indexing of HDF5 files into mainstream use for queries during data analysis and visualization Virtual file driver enhancements Improved Support: Parallel I/O performance tracking, testing and tuning Future Parallel I/O Improvements Copyright © 2010 The HDF Group. All Rights Reserved
www.hdfgroup.org 17 Contract w/LLNL to do: Scalable I/O performance tracking, testing and tuning Virtual file driver enhancements HPC Specific fasttracking Parallel interface enhancements Exploratory design development User support and routine maintenance Future Parallel I/O Improvements Copyright © 2010 The HDF Group. All Rights Reserved
www.hdfgroup.org The HDF Group Performance Hints for Using Parallel HDF5 18 Copyright © 2010 The HDF Group. All Rights Reserved
www.hdfgroup.org 19 Pass along MPI Info hints to file open: H5Pset_fapl_mpio Use MPI-POSIX file driver to access file: H5Pset_fapl_mpiposix Align objects in HDF5 file: H5Pset_alignment Use collective mode when performing I/O on datasets: H5Pset_dxpl_mpio before H5Dwrite/H5Dread Avoid datatype conversions: make memory and file datatypes the same Advanced: explicitly manage metadata flush operations with H5Fset_mdc_config Hints for Using Parallel HDF5 Copyright © 2010 The HDF Group. All Rights Reserved
TWO STEP EQUATIONS 1. SOLVE FOR X 2. DO THE ADDITION STEP FIRST
Info to Enterprise Migration Implementation Case Study: SBC Corporation Presented to the Crystal Decisions Regional Users Group for the Bay Area on October.
Chapter 13: I/O Systems I/O Hardware Application I/O Interface
Advanced Piloting Cruise Plot.
Programming with Android: SDK install and initial setup Luca Bedogni Marco Di Felice Dipartimento di Scienze dellInformazione Università di Bologna.
Chapter 1 The Study of Body Function Image PowerPoint
Processes and Operating Systems
Copyright © 2011, Elsevier Inc. All rights reserved. Chapter 5 Author: Julia Richards and R. Scott Hawley.
Copyright © 2011, Elsevier Inc. All rights reserved. Chapter 4 Author: Julia Richards and R. Scott Hawley.
1 Copyright © 2013 Elsevier Inc. All rights reserved. Appendix 01.
1 Copyright © 2013 Elsevier Inc. All rights reserved. Chapter 3 CPUs.
1 Copyright © 2010, Elsevier Inc. All rights Reserved Fig 2.1 Chapter 2.
1 Copyright © 2013 Elsevier Inc. All rights reserved. Chapter 38.
1 Chapter 40 - Physiology and Pathophysiology of Diuretic Action Copyright © 2013 Elsevier Inc. All rights reserved.
By D. Fisher Geometric Transformations. Reflection, Rotation, or Translation 1.
and 6.855J Cycle Canceling Algorithm. 2 A minimum cost flow problem , $4 20, $1 20, $2 25, $2 25, $5 20, $6 30, $
© 1998, Progress Software Corporation 1 Migration of a 4GL and Relational Database to Unicode Tex Texin International Product Manager.
DCV: A Causality Detection Approach for Large- scale Dynamic Collaboration Environments Jiang-Ming Yang Microsoft Research Asia Ning Gu, Qi-Wei Zhang,
Jeopardy Q 1 Q 6 Q 11 Q 16 Q 21 Q 2 Q 7 Q 12 Q 17 Q 22 Q 3 Q 8 Q 13
© 2018 SlidePlayer.com Inc. All rights reserved.