ADIOS – adiosapi.org1 Jay Lofstead Flexible IO and Integration for Scientific Codes Through The Adaptable IO System (ADIOS) Jay Lofstead (GT),

Slides:

Advertisements

Similar presentations

Hasan Abbasi Matthew Wolf Jay Lofstead Fang Zheng Greg Eisenhauer Karsten Schwan Analyzing large data sets quickly Scott Klasky Ron Oldfield Norbert Podhorszki.

Advertisements

Jialin Liu, Bradly Crysler, Yin Lu, Yong Chen Oct. 15. Seminar Data-Intensive Scalable Computing Laboratory (DISCL) Locality-driven High-level.

University of Chicago Department of Energy The Parallel and Grid I/O Perspective MPI, MPI-IO, NetCDF, and HDF5 are in common use Multi TB datasets also.

A PLFS Plugin for HDF5 for Improved I/O Performance and Analysis Kshitij Mehta 1, John Bent 2, Aaron Torres 3, Gary Grider 3, Edgar Gabriel 1 1 University.

Phillip Dickens, Department of Computer Science, University of Maine. In collaboration with Jeremy Logan, Postdoctoral Research Associate, ORNL. Improving.

Workflow automation for processing plasma fusion simulation data Norbert Podhorszki Bertram Ludäscher Scientific Computing Group Oak Ridge National Laboratory.

Architecture and Implementation of Lustre at the National Climate Computing Research Center Douglas Fuller National Climate Computing Research Center /

HPC User Forum 9/10/08 Managed by UT-Battelle for the Department of Energy 1 HPC User Forum 9/10/2008 Scott Klasky S. Ethier, S. Hodson,

1 Presentation at SciDAC face-to-face January 2005 Ron A. Oldfield Sandia National Laboratories The Lightweight File System.

Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy’s National Nuclear.

I/O Analysis and Optimization for an AMR Cosmology Simulation Jianwei LiWei-keng Liao Alok ChoudharyValerie Taylor ECE Department Northwestern University.

Astrophysics, Biology, Climate, Combustion, Fusion, Nanoscience Working Group on Simulation-Driven Applications 10 CS, 10 Sim, 1 VR.

Connecting HPIO Capabilities with Domain Specific Needs Rob Ross MCS Division Argonne National Laboratory

ADIOS IO introduction Yufei Dec 10. System at Oak Ridge 672 OSTs 10 Petabytes of storage 60 GB/sec = 480 Gbps aggregate performance (theoretical) 225,000.

Scott Klasky SDM Integration Framework in the Hurricane of Data SDM AHM 10/07/2008 Scott A. Klasky ANL: Ross CalTech: Cummings GT: Abbasi,

Tanzima Z. Islam, Saurabh Bagchi, Rudolf Eigenmann – Purdue University Kathryn Mohror, Adam Moody, Bronis R. de Supinski – Lawrence Livermore National.

Alok 1Northwestern University Access Patterns, Metadata, and Performance Alok Choudhary and Wei-Keng Liao Department of ECE,

Parallel HDF5 Introductory Tutorial May 19, 2008 Kent Yang The HDF Group 5/19/20081SCICOMP 14 Tutorial.

Slide 1 Auburn University Computer Science and Software Engineering Scientific Computing in Computer Science and Software Engineering Kai H. Chang Professor.

1 High level view of HDF5 Data structures and library HDF Summit Boeing Seattle September 19, 2006.

HDF5 A new file format & software for high performance scientific data management.

High Performance I/O and Data Management System Group Seminar Xiaosong Ma Department of Computer Science North Carolina State University September 12,

A Metadata Based Approach For Supporting Subsetting Queries Over Parallel HDF5 Datasets Vignesh Santhanagopalan Graduate Student Department Of CSE.

Extending Petascale I/O with Data Services Hasan Abbasi Karsten Schwan Matthew Wolf Jay Lofstead Scott Klasky (ORNL)

CCGrid 2014 Improving I/O Throughput of Scientific Applications using Transparent Parallel Compression Tekin Bicer, Jian Yin and Gagan Agrawal Ohio State.

1 Arie Shoshani, LBNL SDM center Scientific Data Management Center(SDM-ISIC) Arie Shoshani Computing Sciences Directorate Lawrence Berkeley National Laboratory.

Presented by On the Path to Petascale: Top Challenges to Scientific Discovery Scott A. Klasky NCCS Scientific Computing End-to-End Task Lead.

Presented by Reliability, Availability, and Serviceability (RAS) for High-Performance Computing Stephen L. Scott and Christian Engelmann Computer Science.

Supercomputing Cross-Platform Performance Prediction Using Partial Execution Leo T. Yang Xiaosong Ma* Frank Mueller Department of Computer Science.

Accelerating Scientific Exploration Using Workflow Automation Systems Terence Critchlow (LLNL) Ilkay Altintas (SDSC) Scott Klasky(ORNL) Mladen Vouk (NCSU)

SciDAC All Hands Meeting, March 2-3, 2005 Northwestern University PIs:Alok Choudhary, Wei-keng Liao Graduate Students:Avery Ching, Kenin Coloma, Jianwei.

Crystal Ball Panel ORNL Heterogeneous Distributed Computing Research Al Geist ORNL March 6, 2003 SOS 7.

Opportunities in Parallel I/O for Scientific Data Management Rajeev Thakur and Rob Ross Mathematics and Computer Science Division Argonne National Laboratory.

Parallel and Grid I/O Infrastructure W. Gropp, R. Ross, R. Thakur Argonne National Lab A. Choudhary, W. Liao Northwestern University G. Abdulla, T. Eliassi-Rad.

Project 4 : SciDAC All Hands Meeting, September 11-13, 2002 A. Choudhary, W. LiaoW. Gropp, R. Ross, R. Thakur Northwestern UniversityArgonne National Lab.

DOE PI Meeting at BNL 1 Lightweight High-performance I/O for Data-intensive Computing Jun Wang Computer Architecture and Storage System Laboratory (CASS)

1 MosaStore -A Versatile Storage System Lauro Costa, Abdullah Gharaibeh, Samer Al-Kiswany, Matei Ripeanu, Emalayan Vairavanathan, (and many others from.

Presented by End-to-End Computing at ORNL Scott A. Klasky Scientific Computing National Center for Computational Sciences In collaboration with Caltech:

Towards Exascale File I/O Yutaka Ishikawa University of Tokyo, Japan 2009/05/21.

1 Public DAFS Storage for High Performance Computing using MPI-I/O: Design and Experience Arkady Kanevsky & Peter Corbett Network Appliance Vijay Velusamy.

The HDF Group Milestone 5.1: Initial POSIX Function Shipping Demonstration Jerome Soumagne, Quincey Koziol 09/24/2013 © 2013 The HDF Group.

CCGrid 2014 Improving I/O Throughput of Scientific Applications using Transparent Parallel Compression Tekin Bicer, Jian Yin and Gagan Agrawal Ohio State.

ESMF/V3: Managed by UT-Battelle for the Department of Energy.

F. Douglas Swesty, DOE Office of Science Data Management Workshop, SLAC March Data Management Needs for Nuclear-Astrophysical Simulation at the Ultrascale.

DoE I/O characterizations, infrastructures for wide-area collaborative science, and future opportunities Jeffrey S. Vetter, Micah Beck, Philip Roth Future.

Extreme Scale: Programming. Participants Mary Hall Rob Ross Christine Sweeney Ian Foster Daniel Laney Lavanya Ramakrishnan Jim Ahrens John Wright Craig.

Intel Research & Development ETA: Experience with an IA processor as a Packet Processing Engine HP Labs Computer Systems Colloquium August 2003 Greg Regnier.

The HDF Group Introduction to netCDF-4 Elena Pourmal The HDF Group 110/17/2015.

Jay Lofstead Input/Output APIs and Data Organization for High Performance Scientific Computing November.

Presented by Scientific Data Management Center Nagiza F. Samatova Oak Ridge National Laboratory Arie Shoshani (PI) Lawrence Berkeley National Laboratory.

DOE Network PI Meeting 2005 Runtime Data Management for Data-Intensive Scientific Applications Xiaosong Ma NC State University Joint Faculty: Oak Ridge.

SDM Center High-Performance Parallel I/O Libraries (PI) Alok Choudhary, (Co-I) Wei-Keng Liao Northwestern University In Collaboration with the SEA Group.

SDM Center Parallel I/O Storage Efficient Access Team.

Collaborative Scientific Visualization: from your lab to Internet2 and beyond Matthew Wolf College of Computing Georgia Institute of Technology

PIDX PIDX - a parallel API to capture the data models used by HPC application and write it out in an IDX format. PIDX enables simulations to write out.

LIOProf: Exposing Lustre File System Behavior for I/O Middleware

Presented by Robust Storage Management On Desktop, in Machine Room, and Beyond Xiaosong Ma Computer Science and Mathematics Oak Ridge National Laboratory.

Model-driven Data Layout Selection for Improving Read Performance Jialin Liu 1, Bin Dong 2, Surendra Byna 2, Kesheng Wu 2, Yong Chen 1 Texas Tech University.

Towards a High Performance Extensible Grid Architecture Klaus Krauter Muthucumaru Maheswaran {krauter,

Achieving the Ultimate Efficiency for Seismic Analysis

Threads vs. Events SEDA – An Event Model 5204 – Operating Systems.

Spark Presentation.

Locality-driven High-level I/O Aggregation

Adapting Applications and Platforms

Unstructured Grids at Sandia National Labs

Toward a Unified HPC and Big Data Runtime

SDM workshop Strawman report History and Progress and Goal.

A Software-Defined Storage for Workflow Applications

TeraScale Supernova Initiative

In Situ Fusion Simulation Particle Data Reduction Through Binning

Presentation transcript:

ADIOS – adiosapi.org1 Jay Lofstead Flexible IO and Integration for Scientific Codes Through The Adaptable IO System (ADIOS) Jay Lofstead (GT), Scott Klasky (ORNL), Karsten Schwan(GT), Norbert Podhorszki (ORNL), Chen Jin (ORNL) June 23, 2008

ADIOS – adiosapi.org 2Jay Lofstead Outline Motivation Design Goals Architecture Related Work Supported Features Examples Codes & Performance Future Work Collaborators

ADIOS – adiosapi.org 3Jay Lofstead Motivation Multiple HPC architectures Cray, IB-based clusters, BlueGene Many different APIs MPI-IO, POSIX, HDF5, netCDF GTC (fusion) has changed IO routines 8 times so far based on moving platforms Different IO patterns Restarts, analysis, diagnostics Different combinations provide different levels of I/O performance

ADIOS – adiosapi.org 4Jay Lofstead Design Goals ADIOS Fortran and C based API almost as simple as standard POSIX IO External configuration to describe metadata and control IO settings Take advantage of existing IO techniques (no new native IO methods) Fast, simple-to-write, efficient IO for multiple platforms without changing the source code

ADIOS – adiosapi.org 5Jay Lofstead Design Goals ADIOS Contributions: 1.Data Grouping: Multiple, independently controlled IO settings diagnostics, restarts, analysis, viz 2.Optional data items: Different data from different processes Single group write has header from a single proc and data from all 3.Data sizes change dynamically: Datasets vary in size Run size changes data sizes; also AMR codes 4.Constructed output is special: Reused buffers must be handled properly stack temporaries... 5.IO memory is second to science: Buffer space for IO is strictly limited respect the memory needs of the scientific codes

ADIOS – adiosapi.org 6Jay Lofstead Design Goals Provide support for these future features: Provide integration opportunities for other systems Workflow In situ visualization Other simulations Provenance collection Other data management tools Science aware data compression

ADIOS – adiosapi.org 7Jay Lofstead Architecture Thin API XML file data groupings with annotation IO method selection buffer sizes Common tools Buffering Scheduling Pluggable IO routines External Metadata (XML file) Scientific Codes ADIOS API DART LIVE/DataTapMPI-IO POSIX IO HDF-5 pnetCDF Viz Engines Others (plug-in) bufferingschedulefeedback

ADIOS – adiosapi.org 8Jay Lofstead Architecture Data groupings logical groups of related items written at the same time. Not necessarily one group per writing event IO Methods Choose what works best for each grouping Vetted and/or written by experts for each POSIX (Wei-keng Lao Northwestern) MPI-IO (Steve Hodson, ORNL) MPI-IO Collective (Wei-keng Lao Northwestern) NULL (Jay Lofstead GT) Ga Tech DataTap Asynchronous (Hasan Abbasi GT) others..

ADIOS – adiosapi.org 9Jay Lofstead Related Work Specialty APIs HDF-5 – complex API Parallel netCDF – no structure File system aware middleware MPI ADIO layer – File system connection, complex API Parallel File systems Lustre – Metadata server issues PVFS2 – client complexity LWFS – client complexity GPFS, pNFS, Panasas

ADIOS – adiosapi.org 10Jay Lofstead Supported Features Platforms tested Cray CNL (ORNL Jaguar) Cray Catamount (ORNL old-Jaguar and SNL Redstorm) Linux Infiniband (ORNL Ewok) IO Methods MPI-IO independent, MPI-IO collective, POSIX, NULL, Ga Tech DataTap asynchronous, Rutgers DART asynchronous

ADIOS – adiosapi.org 11Jay Lofstead Examples Setup/Cleanup code call adios_init (‘config.xml’)... call adios_finalize (my_rank) adios_init – reads the XML file (once for every process) adios_finalize – provide opportunity for cleanup

ADIOS – adiosapi.org 12Jay Lofstead Examples IO code call adios_open (handle, ‘filename’, ‘groupname’) call adios_write (handle, ‘varname’, var) ADIOS_WRITE(handle,var_name)... call adios_close (handle)

ADIOS – adiosapi.org 13Jay Lofstead Examples IO code call adios_open (handle, ‘filename’, ‘groupname’) ADIOS_GWRITE (handle) call adios_close (handle) - ADIOS_GREAD coming soon

ADIOS – adiosapi.org 14Jay Lofstead Examples IO hints (for asynchronous IO) call adios_end_iteration () pacing hints for asynchronous transport methods use in conjunction with ‘iterations’ attribute of method element in XML call adios_begin_calculation () hint for asynchronous IO that a low-IO phase is starting call adios_end_calculation () hint for asynchronous IO that a low-IO phase is ending

ADIOS – adiosapi.org 15Jay Lofstead Examples XML file contents (data elements) <var name=“data” type=“double” path=“/” dimensions=“elements” copy-on-write=“yes”/>

ADIOS – adiosapi.org 16Jay Lofstead Examples XML file contents (other) params

ADIOS – adiosapi.org 17Jay Lofstead Codes & Performance Codes GTC (fusion), Chimera (supernova), GTS (fusion), XGC0 (fusion), XGC1 (fusion), Flash (astrophysics), S3D (combustion) Performance Measurements GTC: nearly 25 GB/sec GTS: 12 GB/sec Chimera: 6.5% reduction in wall clock time

ADIOS – adiosapi.org 18Jay Lofstead Codes & Performance

ADIOS – adiosapi.org 19Jay Lofstead Codes & Performance Old GTC IO New GT ADIOS/DataTap IO 12 GB/restart GTC On Rizzo at ORNL

ADIOS – adiosapi.org 20Jay Lofstead Codes & Performance June 7, 2008: 24 hour GTC run on Jaguar at ORNL 93% of machine (28,672 cores) MPI-OpenMP mixed model on quad-core nodes (7168 MPI procs) three interruptions total (simple node failure) with hour runs Wrote 56 TB of data at 20 GB/sec (21 TB for post analysis) IO overhead 3% of wall clock time Mixed IO methods of synchronous MPI-IO and POSIX IO configured in the XML file

ADIOS – adiosapi.org 21Jay Lofstead Support Tools bpdump – similar to h5dump bp2h5 – convert output file to HDF5 bp2nc – convert output to netCDF

ADIOS – adiosapi.org 22Jay Lofstead Future Work Work on read functionality Index files Additional native transports direct parallel HDF5 and parallel netCDF asynchronous MPI Visit others as demanded

ADIOS – adiosapi.org 23Jay Lofstead Future Work Offload to staging area for other processing

ADIOS – adiosapi.org 24Jay Lofstead Collaborators Chen Jin, Scott Klasky, Stephen Hodson, James B. White III, Weikuan Yu (Oak Ridge National Laboratory) Hasan Abbasi, Greg Eisenhauer, Karsten Schwan, Matthew Wolf, Fang Zheng (Georgia Tech) Wei-keng Liao, Alok Choudhary, Northwestern University Manish Parashar, Ciprian Docan, Rutgers University. Ron Oldfield, Sandia Labs

ADIOS – adiosapi.org 25Jay Lofstead References Jay Lofstead, Scott Klasky, Karsten Schwan, Norbert Podhorszki, Chen Jin. “Flexible IO and Integration for Scientific Codes Through The Adaptable IO System (ADIOS).” In CLADE 2008 at HPDC, Boston, MA, June ACM. Hasan Abbasi, Matthew Wolf, Karsten Schwan. “Live data workspace: A flexible, dynamic and extensible platform for petascale applications.” In Cluster Computing, Austin, TX, September IEEE International. Chen Jin, Scott Klasky, Stephen Hodson, Weikuan Yu (Oak Ridge National Laboratory) Jay Lofstead, Hasan Abbasi, Karsten Schwan, Matthew Wolf (Georgia Tech) Wei-keng Liao, Alok Choudhary(Northwestern University) Manish Parashar, Ciprian Docan, Rutgers University. Ron Oldfield (Sandia National Laboratories). “Adaptive IO System (ADIOS).” In CUG Helsinki, Findland, May Sarp Oral. NCCS Jaguar GTC 128 Nodes Test Results.