1. Bologna, 21 Marzo 2006 Riccardo Zappi / Luca Magnoni INFN-CNAF, Bologna

2. Bologna, 21 Marzo 2006 INFN Storage Workshop Page 2 Summary 1. What SRM is 2. What StoRM is 3. StoRM status 4. StoRM next release 5. Tests done on StoRM 6. Conclusion

3. Bologna, 21 Marzo 2006 INFN Storage Workshop Page 3 What SRM is SRMs are middleware components that manage shared storage resources on the grid and provide management functionalities like: –Uniform access to heterogeneous types of storage –File pinning –Disk space allocation and advanced disk space reservation –Protocol negotiation –Life time management of files –Management of security SRMs implements a SRM interface specified by GSM-WG within GGF. (v.1.1, v.2.1.1 and v.3.0)

4. Bologna, 21 Marzo 2006 INFN Storage Workshop Page 4 Managent Access & Data Access to SE

5. Bologna, 21 Marzo 2006 INFN Storage Workshop Page 5 GFAL –a low-level IO interface to Grid Storage –Storage Element Abstraction ( SRM v1) lcg_util –Provides both direct C library calls and CLI tools –Is a thin wrapper on top of GFAL SRM v1 FTS –The FTS receive requests either through their Web Service interface directly from the clients or indirectly through the Data Scheduler. –The transfers managed by the FTS are all asynchronous, i.e. the client submits a transfer job, which may contain a list of files (with proper source and destination qualifiers) to be transferred. –Current version use SRM v1. How SRM-service should be used?

6. Bologna, 21 Marzo 2006 What StoRM is

7. Bologna, 21 Marzo 2006 INFN Storage Workshop Page 7 What StoRM is StoRM is a Storage Resource Manager. It is exposed as web service. –Web service description (wsdl) of StoRM is compliant with SRM specification version 2.1.1 (WS-I compliant). StoRM uses Cluster File System with ACL support. Current StoRM version is built on top of GPFS StoRM allows direct access (native posix call) –StoRM works without server I/O too. –Aggregation is performed by underlying file system (better than StoRM could do).

8. Bologna, 21 Marzo 2006 INFN Storage Workshop Page 8 StoRM is different … Gli accessi allo storage avvengono con modalità Posix-like. Linterfaccia esposta NON è del File System, ma è un wrapping fatto attraverso qualche libreria.. (dcap, rfio, ecc.) Gli accessi allo storage avvengono attraverso chiamate POSIX esposte dal file system sottostante. Nel caso di Parallel File System, si possono sfruttare in pieno le performance offerte!! I metadati relativi ai file e allo storage (come il namespace, space management, ecc..) sono gestiti completamente a livello applicativo, replicando le funzionalità tipiche dei file system (performance? Scalabilita?) I metadati relativi ai file e allo storage, (quando possibile), sono gestiti completamente dal file system sottostante. Si evita così di replicare le stesse funzionalità del file system a livello applicativo (performance! Scalabilita!)

9. Bologna, 21 Marzo 2006 INFN Storage Workshop Page 9 Front end (FE) has responsibilities of : –expose a web service interface –manage connection with authorized clients –store asynchronous request into data base. –retrieve asynchronous request status. –co-operate with backend directly for synchronous call. –co-operate with external authorization service to enforce security policy on service. –manage user authentication Data Base : –Store SRM request and status –Store application data Back end (BE) has responsibilities of : –accomplish all synchronous (active) action. –get asynchronous request from data base. –accomplish all asynchronous action. –bind with underlying file system. –enforce authorization policy on files –manage SRM file and space metadata. StoRM architecture

10. Bologna, 21 Marzo 2006 INFN Storage Workshop Page 10 StoRM Security Aspects 1.User perform srmPrepareToGet 2.StoRM verifies if the principal holds a valid proxy certificate and delegates the external policy decision point to validate the request. 3.StoRM then queries the Authorization Sources to verify if the user can perform the specified operation on the SURL 4.StoRM queries LCMAPS to obtain local user account corresponding to the grid identity of the requestor 5.Physical file name derives by SURL and user attributes (Virtual organization name space) 6.The file system wrapper enforces permissions by setting a new ACL on the physical file. 7.The user job can be executed into the worker node 8.The application can perform a standard POSIX call to access the file into/from the storage system.

11. Bologna, 21 Marzo 2006 INFN Storage Workshop Page 11 Current release provides Command Line Clients. These Clients are intended for Test use only. FTS, lcg-util and GFAL are not usable with SRM v.2.1 yet. Interoperability Clients are going to be used with StoRM. StoRM Command Line clients

12. Bologna, 21 Marzo 2006 StoRM : Status 1)Current release 2)Next release

13. Bologna, 21 Marzo 2006 INFN Storage Workshop Page 13 StoRM status – Release First release [v 1.0.0] –End of November 2005 : base version Christmas release [v 1.0.1] –Xmas 2005 : patched version Second release [v 1.0.2] –End of January 2006 (CHEP version) : directory functions. Current release provides these functionalities: -Data transfer : srmCopy, srmPtG, srmPtP, srmStatus -Space Management : srmReserveSpace, srmGetSpaceMetadata -Directory : srmLs, srmRm, srmMkDir, srmRmdir. Documentation (installation and configuration) available. Web Site of project : User management to better support

14. Bologna, 21 Marzo 2006 INFN Storage Workshop Page 14 Third release [v 1.1.0] pre INFN storage workshop Build completed ( Friday 17 March..) Important improvements implemented New clients –Interoperability (WSDL WS-I compliant, SRM v.2.1.1_modified) –Inline help improved –Parameter syntax checking Front End improved –WSDL WS-I compliant –Larger space reservation (2^64 byte) –Better explanation in case of failure Back End improved –Space Reservation : multiple utilization –thread-safe LCMAPS usage –performance tuning StoRM status – Today

15. Bologna, 21 Marzo 2006 INFN Storage Workshop Page 15 StoRM status – fine Aprile/Maggio Fourth release [v 1.2.0] Scheduled for May (check web site for news). Important improvements designed: Client improvement –More parameters available –Client-API on space management for Advance Reservation service. Front End improvement –VOMS – GPBOX authorization service integration –Checking on malformed parameters –Information Service Integration Back End improvement –Configuration file based on XML format ( hierarchic and more expressive) –Authorization source integration (LFC and/or ad-hoc solution) –New functionalities : service monitoring, … Web site and documentation –New documentation (installation and configuration) in different media (wiki).

16. Bologna, 21 Marzo 2006 StoRM : TEST

17. Bologna, 21 Marzo 2006 INFN Storage Workshop Page 17 StoRM TESTING – current version Third release [v 1.1.0] Site A :Tier1 (22TB) Site B :Bari (2TB) Site C :ICTP-Trieste (30GB) Site D :CNAF-Cert-SE (50GB) Tests Functionality tests : Sito C e D Stress tests : Sito A Transfer tests : Siti A B

18. Bologna, 21 Marzo 2006 INFN Storage Workshop Page 18 Test results Data transfer T1 to/from Bari via srmCopy v.2.1.1 50 parallel srmCopy with From SURL at CNAF To SURL at BARI 1GB file size, everyone Bandwidth saturated. Only 100Mb/s

19. Bologna, 21 Marzo 2006 StoRM : Releasing process Group dedicated to Developing Documentation Testing and integration Prototyping

20. Bologna, 21 Marzo 2006 INFN Storage Workshop Page 20 Releasing process

21. Bologna, 21 Marzo 2006 INFN Storage Workshop Page 21 StoRM service development involves many aspects (Development, Integration, Testing, Prototyping, Dissemination). Important progress done on each service component. StoRM will be in production version by June! We need help on Testing and Prototyping. Help on integration is welcome. Conclusions

22. Bologna, 21 Marzo 2006 INFN Storage Workshop Page 22 Alberto Forti (CNAF), from 01 April Luca Magnoni (CNAF), 100% Riccardo Zappi (CNAF), 100% Ezio Corso (ICTP), 100% Antonio Messina (ICTP), 50% Riccardo Murri (ICTP), 50% Project managed by Antonia Ghiselli. StoRM Develop Team

23. Bologna, 21 Marzo 2006 INFN Storage Workshop Page 23 Questions and Suggestions ?!