1. StoRM: current status and developments
Michele Dibenedetto
StoRM Product Team
INFN-CNAF
19 May 2011, Workshop CCR INFN GRID, La Biodola

2. Overview
Before EMI
In EMI 1
In EMI 2
EMI 1.X – between the releases
Outside EMI duties
Conclusions

3. Overview
Before EMI
In EMI 1
In EMI 2
EMI 1.X – between the releases
Outside EMI duties
Conclusions

4. Releasing Current version 1.6.2
platform supported is SL5 x86_64
Distributed within INFN-GRID (IG) releases
RPM available from IG repositories
This is the last StoRM release within IG
Configured via yaim
needs glite-yaim-core, ig-yaim and ig-yaim-storm components
Provides 4 profiles to configure 4 different components possibly on 4 different hosts

5. IG StoRM 1.6 deployment scenario

6. Overview
Before EMI
In EMI 1
In EMI 2
EMI 1.X – between the releases
Outside EMI duties
Conclusions

7. What’s new: HTTP(s) support
A new component comes with StoRM release: StoRM GridHttps Server
supports http and https file access protocols
http: allows everyone to read and write given a valid TURL
no user authentication required
https: performs user authentication to enforce request authorization
requires a valid user proxy
behaves as a PEP for read/write user requests
checks the rights of the proxy owner by using StoRM Backend as PDP
supports proxy with VOMS extensions

8. What’s new: GLUE 2.0
A common GLUE 2.0 schema interpretation has been reached by EMI JRA1 data task force1
Data are published following both GLUE 1.3 and GLUE 2.0 schemas
Currently GLUE 1.3 information is published on both the schemas
1 -

9. What’s new: Dynamic Info Provider
Revised StoRM Dynamic Info Provider
storage information retrieved contacting StoRM Backend
query key is the Space Alias
new REST interface to query storage information (JSON) exposed by StoRM Backend
Storage information rendered from JSON format to LDIF format.

10. What’s new: installation
Installation paths have changed
StoRM components follow Filesystem Hierarchy Standard (FHS) guidelines
StoRM components dependencies are resolved from standard repositories
SL5
openldap, mysql-server, ant, tomcat, jakarta-X, …
EPEL
boost, globus-X, gsoap, bouncycastle, …
EMI repository
lcmaps, voms, lcas, trustmanager, yaim-core, …

11. Enhancements
Added the production of a binary tar.gz upon all the StoRM component builds
Provided a new installation guide with the adoption of the EMI guidelines rules
Adopted the EMI software and documentation quality assurance policies
Adopted a new product Issue Tracker (Redmine)
Adopted a new certification process2
2 - E. Ronchieri, M. Dibenedetto, R. Zappi, S. Dal Pra, C. Aiftimiei, S. Traldi: The StoRM Certification Process, CHEP 2010, October , Taipei, Taiwan.

12. StoRM backend: Enhancements
Added some sanity checks on StoRM backend startup to verify filesystem related issues
Storage Area’s mount points are verified
A clear message appears in the log file if your EXT3 file system is declared a GPFS filesystem in namespace.xml !
ACL and extended attributes capabilities are verified
Refactored REST services definition module
Added a distinct automatic update tool for tape recall database

13. StoRM backend: Bug fixes
Fixed a bug in storage space consuption update upon transfer completion
Fixed a bug on tape recall to manage simultaneous recall request on the same file
Fixed a memory leak in tape recall request caching

14. Releasing New version StoRM 1.7 Distributed within EMI releases
platform supported is SL5 x86_64
Distributed within EMI releases
No plans for supporting the upgrade from non-EMI releases to StoRM 1.7
First EMI release announced on 12/05/2011
but StoRM has not ben released (next slide)

15. Why StoRM 1.7 is not in EMI 1
StoRM Frontend component cannot be released
has shown incopatibility with new gsoap version
StoRM certification tests suite cannot be run
the missing component prevent from testing the whole
Decided to use StoRM Frontend to test StoRM 1.7
Next EMI bug fix release will contain StoRM apart from the Frontend

16. Why StoRM 1.7 is not in EMI 1
We are currently investigating how to solve gsoap issue
A new EMI bug fix release will contain all StoRM components

17. Configuration Still configured with yaim but with new components
needs glite-yaim-core and storm-yaim
Provides 5 profiles to configure 5 different components possibly on 5 different machines

18. EMI StoRM 1.7 deployment scenario

19. Overview
Before EMI
In EMI 1
In EMI 2
EMI 1.X – between the releases
Outside EMI duties
Conclusions

20. What’s in plans Scheduled for M24 (May 2012)
WEBDAV file access facilities
Server side support for File Catalog and Storage Element synchronization
Publish GLUE 2.0 information

21. Overview
Before EMI
In EMI 1
In EMI 2
EMI 1.X – between the releases
Outside EMI duties
Conclusions

22. What’s in plans StoRM bug fix releases will be produced when needed
The procedure is regulated by EMI
provide user blacklisting via Argus integration
StoRM Frontend technology switch: have a JAVA implementation
leverage on existing horizontal scaling solutions
provide a simple way to achieve monitoring (JMX)
take advantage from StoRM Product Team JAVA skills to increase Frontend reliability and efficiency

23. Overview
Before EMI
In EMI 1
In EMI 2
EMI 1.X – between the releases
Outside EMI duties
Conclusions

24. User’s requirements: administrators
Increase StoRM Backend reliability and robustness
Increase StoRM deployment scenario flexibility
Let StoRM scale better
Let’s have StoRM Backends pool!

25. Backend pool scenario

26. User’s requirements: administrators
StoRM components produce many log information
Each has at least one log file
Information scattered in the filesystem, each with its own formalism
Possibly on different hosts
System administrators hell!

27. User’s desiderata Let’s provide an integrated solution :
single point log aggregator
only relevant information extracted from huge log files
related events from different log sources rendered in a common format
providing a searchable history of events with different levels of aggregation
Let’s have StoRMON3!
3 – E. Ronchieri, M. Dibenedetto, R. Zappi, S. Dal Pra, C. Aiftimiei, S. Traldi: StoRMon: an event log analyzer for Grid Storage Element based on StoRM, CHEP 2010, October , Taipei, Taiwan.

28. Backend pool scenario

29. User’s desiderata: SRM users
Perform SRM prepare-to-get and prepare-to-put synchronously
Perform space accounting
...
All this development is performed best effort
JAVA developers needed

30. Overview
Before EMI
In EMI 1
In EMI 2
EMI 1.X – between the releases
Outside EMI duties
Conclusions

31. New releasing channels, RPM repositories and yaim node-types
Conclusions
EMI 1 task accomplished
but not yet released
New component
New releasing channels, RPM repositories and yaim node-types
New standards adopted
Bug fixes and enhancements
Plans for increase stability and scalability

32. Thank you!