StoRM: a SRM solution for disk based storage systems Magnoni Luca INFN-CNAF, Bologna luca.magnoni@cnaf.infn.it

Outline StoRM Conclusions and References StoRM Team Architecture Grid usage scenario Security Framework SRM issues Deployment Conclusions and References StoRM Team T1-T2 Workshop

StoRM StoRM is a storage resource manager for disk based storage systems. It implements the SRM interface version 2.x. StoRM is designed to support guaranteed space reservation and direct access (native POSIX I/O call), as well as other standard libraries (like RFIO). StoRM take advantage from high performance parallel file systems. Also standard POSIX file systems are supported. A modular architecture decouples StoRM logic from the supported file system. Strong security framework with VOMS support. T1-T2 Workshop

StoRM architecture Front end (FE): Data Base (DB): Back end (BE): Exposes the web service interface. Manages user authentication. Sends the request to the BE. Data Base (DB): Stores SRM request and status. Stores file and space information. Back end (BE):  Executes all synchronous and asynchronous functionalities. Binds with the underlying file systems. Enforces authorization policy on files. Manages SRM file and space metadata. In this slide you can see an artistic view of StoRM architecture. T1-T2 Workshop

StoRM General Considerations 1/2 File system currently supported by StoRM GPFS from IBM. XFS from SGI. Any other File System with POSIX interface and ACLs support. Light and flexible namespace structure The namespace of the files managed by StoRM relies upon the underlying file systems. StoRM does not need to query any DB to know the physical location of a requested SURL. T1-T2 Workshop

StoRM General Considerations 2/2 ACLs Usage StoRM enforce ACL entries on physical files for the local user corresponding to the grid- credential. Standard grid applications (such as GridFTP, RFIO, etc.) can access the storage on behalf of the user. Scalability and high availability. FE, DB, and BE can be deployed in 3 different machines. StoRM is designed to be configured with n FE and m BE, with a common DB. But more tests are needed to validate this scenario. T1-T2 Workshop

StoRM Grid usage scenario StoRM dynamically manages files and space in the storage system. Applications can directly access the Storage Element (SE) during the computational process. File metadata are managed (and stored) by underlying file system. No replica of metadata at application level.. That is a file system job! In this way StoRM gain in performance. Data access is performed without interacting with an external service, with great performance improvement (POSIX calls). Otherwise, standard data access using I/O Server (such as RFIO) is also fully supported. T1-T2 Workshop

StoRM security framework The user wants to access a file. StoRM verifies if the principal holds a valid proxy certificate and delegates the external policy decision point to validate the request. StoRM then queries the Authorization Sources to verify if the user can perform the specified operation on the file. StoRM queries the Mapping Service to obtain the local user account corresponding to the grid identity of the requester. The file system wrapper enforces permissions by setting a new ACL on the physical file. The user job can be executed into the worker node. The application can perform a standard POSIX call to access the file from the storage system. T1-T2 Workshop

StoRM status and SRM issues Migration to SRM v2.2 completed. All functions requested by the SRM WLCG usage agreement are implemented. New version of StoRM available. StoRM SRM tests StoRM is involved in interoperability tests made by SRM-WG, the results are available here: http://sdm.lbl.gov/srm-tester/v22-progress.html StoRM is involved also in other SRM tests made with S2 test suite: http://gdrb02.cern.ch:25000/srms2test/scripts/protos/sr m/2.2/basic/s2_logs/ T1-T2 Workshop

StoRM deployment StoRM service can be deployed with : RPMs. Yaim. quattor (the profile is under test). Existing StoRM installations: 1 StoRM@Padova with SRM v.2.1 in production for EGRID group. 1 StoRM@Trieste(ELETTRA) with SRM v.2.1 for GridCC. 2 StoRM@Legnaro with SRM v.2.2 for GridCC. 1 StoRM@CNAF with SRM v.2.2 for SRM-WG interoperability test (on the GPFS v.3.1 testbed). 1 StoRM@CNAF with SRM v.2.2 installation in a pre-production environment that will be available for VOs tests (on the GPFS v.2.3 production cluster). We provide also Command Line SRM v2.2 clients. T1-T2 Workshop

Conclusions We presented StoRM, a SRM solution for disk based storage systems. It leverages parallel file systems advantages in a Grid scenario. It is a lightweight SRM implementation for standard POSIX file systems. It provides a strong security framework. T1-T2 Workshop

StoRM is a result from collaboration between: StoRM Team StoRM is a result from collaboration between: INFN – CNAF within the High Energy Physics community CNAF team: Coordinator: A.Ghiselli. Members: A.Forti, L.Magnoni and R.Zappi. ICTP – EGRID Grid infrastructure for Economics and Finance research EGRID team: Coordinators: A.Nobile and S.Cozzini. Members: E.Corso, A.Messina and A.Terpin. T1-T2 Workshop

You can find further information at: StoRM References You can find further information at: http://storm.forge.cnaf.infn.it T1-T2 Workshop

StoRM Tests Performance and stress test were done last spring on a dedicated testbed with 40TB of storage using GPFS v. 2.3 and StoRM implementing the SRM v. 2.1. Large number of requests (1000-10000) performed with different submission rate (10 - 30 requests per second). Brief summary: Sustained read and write throughputs measured on the GPFS cluster. Small number of request failure. Good execution rate. For more information: http://storm.forge.cnaf.infn.it/doku.php?id=documentation:presentatio ns_and_papers T1-T2 Workshop

SRM service interaction These services interact with SRM servers to provide facilities for Grid data access. GFAL Presents a high level interface for the I/O operations. Supported protocols are: file, dcap, rfio. A SRM v2.2 compliant release is ready for testing. lcg_util Provides a thin wrapper (though C library or CLI tools) on top of GFAL. FTS The File Transfer Service. A SRM v.2.2 compliant release should be available from December. T1-T2 Workshop