SDM Center February 2, 2005 Progress on MPI-IO Access to Mass Storage System Using a Storage Resource Manager Ekow J. Otoo, Arie Shoshani and Alex Sim Lawrence Berkeley National Laboratory

SDM Center February 2, 2005 Objective To allow near-online transparent access to files on mass storage system (e.g., HPSS) from MPI applications. To enable existing applications to dynamically access files from MSS with little modification to the source code. To demonstrate the feasibility of the approach by running some applications on a Linux cluster using  PVFS as the local parallel file system and  HPSS as the mass storage system

SDM Center February 2, 2005 MPI Application on a Cluster Accessing an MSS ION Proxy + Svr CN Other MSS Parallel File System GridFTP CN – Compute Node ION – IO Node SRM-Proxy SRM-Svr – SRM Server Mass Storage System (MSS)

SDM Center February 2, 2005 MPI-IO-SRM Architecture Data Intensive Applications MPI-IOSRM PVFSGPFSUFSXFSOther BDBOther ADIO HPSS SAM Jasmin Castor High Level Access and Control Record Structured File Access Low Level File System Access pNetCDF HDF5

SDM Center February 2, 2005 Main Functions of MpiioSrm Library This is a package of library functions called libMpiioSrm.a  The functions callable from MPI applications are: 1.MPI_Srm_proxy_init(); 2.MPI_Srm_file_open(); [in place of MPI_File_open()] 3.MPI_Srm_file_close(); [in place of MPI_File_close()] 4.MPI_Srm_file_delete(); [in place of MPI_File_delete()] 5.MPI_Srm_proxy_close() Functions (2), (3) and (4) are the only ones that take file names as a parameter of all the MPI-IO library functions. These can be used to build complex execution models of an application.

SDM Center February 2, 2005 Dependent SRM Functions Functions equivalent to SRM commands are:  srm_abort(); srm_copy();  srm_get();srm_ls();  srm_mkdir();srm_ping();  srm_put();srm_release();  srm_remove();srm_status(); Parameters used in these functions are either derived from a configuration file “hrm.rc” or communicated through MPI_Info object.

SDM Center February 2, 2005 How to Use the Library Functions Requires inclusion of MpiioSrm.h for compilation and libMpiioSrm during linkage. After MPI_init() we call  MPI_Srm_proxy_init( MPI_Comm comm, int myrank, int srm_enabled, char *proxy_host, int proxy_rank, char *fileofurls, char fileofurl_fmt, char *srmuserid, char *hrmrcpath, MPI_Info *srminfo) ; The application then calls  MPI_Srm_file_open(), …   MPI_Srm_file_close(), etc. Finally it calls  MPI_Srm_proxy_close(MPI_Comm comm, int proxy_rank) before calling MPI_Finalize();

SDM Center February 2, 2005 Some Special Features of MpiioSrm Methods of specifying data sources and targets.  Simple file name – a string of characters of a directory path and basename  URLs  Indirectly through a file of either List of source and target URLs or List of SRM commands Modes for opening files:  In addition to the various MPI_MODE_* of the standard we add the mode MPI_MODE_PROPCHG_ON_CLOSE to propagate any changes on the local file to the MSS. An application can set modes through MPI_File_info_set() command in our case.

SDM Center February 2, 2005 Status of the MpiioSrm library - 1 The current status of the library includes  For SRM services, we have srm_abort(), srm_copy(), srm_ls(), srm_ping(), srm_status(), srm_release() and srm_remove().  For MPI-IO calling SRMs we have all five basic functions implemented, namely MPI_Srm_proxy_init(); MPI_Srm_file_open(); MPI_Srm_file_close(); MPI_Srm_file_delete(); MPI_Srm_proxy_close()

SDM Center February 2, 2005 Status of the MpiioSrm library - 2 Test Programs  The current status includes the following test programs: TestSRM_S1RC – Static Single Request at a time. TestSRM_SRSC – Static Requests with Single file processing at a time. TestSRM_DRSOC – Dynamic Requests with Sequential Processing Order. TestSRM_DRAOC – Dynamic Requests with Arbitrary Processing order.

SDM Center February 2, 2005 Looking to the Future Short Term  Add srm_put() and srm_get() and srm_mkdir()  Improve method of communicating prefetched file with processes using MPI’s Remote Memory Access.  Release for general use. Long Term  Address scalability, fault tolerance issues and PVFS2 – use of multiple SRM servers.  Define XML schema for input file formats  GUI for: Preparing input data files for applics Monitor files fetched and processed by applications through MPI’s Dynamic Process Management functions