Parallel I/O for Distributed Applications (MPI-Conn-IO) SPP Tools Users of MPI-2 Parallel I/O and parallel HDF5 with distributed applications Objectives MPI-2 parallel I/O subsystem that functions across machines Set of management tools for handling files spread across multiple machines Impact and DoD Relevance Interoperation between different vendors’ MPI-2 I/O implementations and optimmization of inter-machine file transfer will achieve high performance remote access to large-scale data sets from distributed applications. PET Team: G. Fagg and S. Browne, University of Tennessee-Knoxville Methodology: The MPI-Connect system allows intercommunication between different MPI implementations using normal MPI communication calls. The MPI-Conn-IO extension to MPI-Connect supports shared access to remote files by parallel and distributed applications. The MPI-Conn-IO extension can be used either with MPI-Connect, in which case MPI-2 I/O calls are intercepted through the MPI profiling interface and dealt with appropriately, or in a stand-alone manner by non-MPI applications. Results: The MPI-Conn-IO client library provides collective calls for reading and writing remote files. Tasks may request either replicated copies or partitioned views of a file. The file transfer protocol attempts to use network bandwidth efficiently by means of blocking, caching, and prefetching. Issues needing further study (e.g., consistency and security requirements) have been identified. Significance: Performance of istributed DoD applications that access large-scale data sets will benefit from the MPI-Conn-IO use of tuned vendor versions of MPI-2 IO on a single platform and of prefetching and caching for multi-platform file access. MPI-Conn-IO support for MPI-2 IO and parallel HDF5 will help ensure portability of distributed applications that use those interfaces. References: S. Browne and G. Fagg, “A Parallel I/O Subsystem for Distributed Applications”, University of Tennessee Computer Science Technical Report (in preparation, to be submitted for publication)