Status of the SRM 2.2 MoU extension

Slides:



Advertisements
Similar presentations
J Jensen CCLRC RAL Data Management AUZN (mostly about SRM though) GGF 16, Athens J Jensen.
Advertisements

LCG Tiziana Ferrari - SC3: INFN installation status report 1 Service Challenge Phase 3: Status report Tiziana Ferrari on behalf of the INFN SC team INFN.
DPM Name Server (DPNS) Namespace Authorization Location of physical files DPM Server Requests queuing and processing Space Management SRM Servers v1.1,
EGEE is a project funded by the European Union under contract IST Using SRM: DPM and dCache G.Donvito,V.Spinoso INFN Bari
Storage: Futures Flavia Donno CERN/IT WLCG Grid Deployment Board, CERN 8 October 2008.
Storage Issues: the experiments’ perspective Flavia Donno CERN/IT WLCG Grid Deployment Board, CERN 9 September 2008.
CASTOR SRM v1.1 experience Presentation at SRM meeting 01/09/2004, Berkeley Olof Bärring, CERN-IT.
LHCC Comprehensive Review – September WLCG Commissioning Schedule Still an ambitious programme ahead Still an ambitious programme ahead Timely testing.
INFSO-RI Enabling Grids for E-sciencE SRMv2.2 experience Sophie Lemaitre WLCG Workshop.
DPM CCRC - 1 Research and developments DPM status and plans Jean-Philippe Baud.
Status report on SRM v2.2 implementations: results of first stress tests 2 th July 2007 Flavia Donno CERN, IT/GD.
SRM 2.2: tests and site deployment 30 th January 2007 Flavia Donno, Maarten Litmaath IT/GD, CERN.
SRM 2.2: status of the implementations and GSSD 6 th March 2007 Flavia Donno, Maarten Litmaath INFN and IT/GD, CERN.
Data Management The GSM-WG Perspective. Background SRM is the Storage Resource Manager A Control protocol for Mass Storage Systems Standard protocol:
CERN IT Department CH-1211 Geneva 23 Switzerland t Storageware Flavia Donno CERN WLCG Collaboration Workshop CERN, November 2008.
D C a c h e Michael Ernst Patrick Fuhrmann Tigran Mkrtchyan d C a c h e M. Ernst, P. Fuhrmann, T. Mkrtchyan Chep 2003 Chep2003 UCSD, California.
Enabling Grids for E-sciencE Introduction Data Management Jan Just Keijser Nikhef Grid Tutorial, November 2008.
Maarten Litmaath (CERN), GDB meeting, CERN, 2006/02/08 VOMS deployment Extent of VOMS usage in LCG-2 –Node types gLite 3.0 Issues Conclusions.
Light weight Disk Pool Manager experience and future plans Jean-Philippe Baud, IT-GD, CERN September 2005.
Owen SyngeTitle of TalkSlide 1 Storage Management Owen Synge – Developer, Packager, and first line support to System Administrators. Talks Scope –GridPP.
1 User Analysis Workgroup Discussion  Understand and document analysis models  Best in a way that allows to compare them easily.
INFSO-RI Enabling Grids for E-sciencE Enabling Grids for E-sciencE Pre-GDB Storage Classes summary of discussions Flavia Donno Pre-GDB.
WLCG Grid Deployment Board, CERN 11 June 2008 Storage Update Flavia Donno CERN/IT.
WebFTS File Transfer Web Interface for FTS3 Andrea Manzi On behalf of the FTS team Workshop on Cloud Services for File Synchronisation and Sharing.
1 LHCb File Transfer framework N. Brook, Ph. Charpentier, A.Tsaregorodtsev LCG Storage Management Workshop, 6 April 2005, CERN.
Maarten Litmaath (CERN), GDB meeting, CERN, 2006/06/07 SRM v2.2 working group update Results of the May workshop at FNAL
Jens G Jensen RAL, EDG WP5 Storage Element Overview DataGrid Project Conference Heidelberg, 26 Sep-01 Oct 2003.
Derek Ross E-Science Department DCache Deployment at Tier1A UK HEP Sysman April 2005.
INFSO-RI Enabling Grids for E-sciencE Introduction Data Management Ron Trompert SARA Grid Tutorial, September 2007.
Busy Storage Services Flavia Donno CERN/IT-GS WLCG Management Board, CERN 10 March 2009.
The CMS Top 5 Issues/Concerns wrt. WLCG services WLCG-MB April 3, 2007 Matthias Kasemann CERN/DESY.
WLCG Grid Deployment Board CERN, 14 May 2008 Storage Update Flavia Donno CERN/IT.
SRM-2 Road Map and CASTOR Certification Shaun de Witt 3/3/08.
Handling of T1D0 in CCRC’08 Tier-0 data handling Tier-1 data handling Experiment data handling Reprocessing Recalling files from tape Tier-0 data handling,
ASCC Site Report Eric Yen & Simon C. Lin Academia Sinica 20 July 2005.
CMS: T1 Disk/Tape separation Nicolò Magini, CERN IT/SDC Oliver Gutsche, FNAL November 11 th 2013.
Grid Deployment Board 5 December 2007 GSSD Status Report Flavia Donno CERN/IT-GD.
LHCC Referees Meeting – 28 June LCG-2 Data Management Planning Ian Bird LHCC Referees Meeting 28 th June 2004.
1 SRM v2.2 Discussion of key concepts, methods and behaviour F. Donno CERN 11 February 2008.
The Grid Storage System Deployment Working Group 6 th February 2007 Flavia Donno IT/GD, CERN.
SRM 2.2: experiment requirements, status and deployment plans 6 th March 2007 Flavia Donno, INFN and IT/GD, CERN.
Bologna, March 30, 2006 Riccardo Zappi / Luca Magnoni INFN-CNAF, Bologna.
Security recommendations DPM Jean-Philippe Baud CERN/IT.
CERN IT Department CH-1211 Genève 23 Switzerland t DPM status and plans David Smith CERN, IT-DM-SGT Pre-GDB, Grid Storage Services 11 November.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks SE Security Rémi Mollon, Ákos Frohner EGEE'08,
Nov 05, 2008, PragueSA3 Workshop1 A short presentation from Owen Synge SA3 and dCache.
Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland t DBES Data Management and Storage Review and progress Andrea Sciabà.
EGEE Data Management Services
Jean-Philippe Baud, IT-GD, CERN November 2007
StoRM: a SRM solution for disk based storage systems
The lightweight Grid-enabled Disk Pool Manager (DPM)
Flavia Donno, Jamie Shiers
Future of WAN Access in ATLAS
Flavia Donno CERN GSSD Storage Workshop 3 July 2007
GDB 8th March 2006 Flavia Donno IT/GD, CERN
Berkeley Storage Manager (BeStMan)
SRM v2.2 / v3 meeting report SRM v2.2 meeting Aug. 29
SRM Developers' Response to Enhancement Requests
SRM2 Migration Strategy
T-StoRM: a StoRM testing framework
Storage Resource Reporting Proposal
GFAL 2.0 Devresse Adrien CERN lcgutil team
Ákos Frohner EGEE'08 September 2008
The INFN Tier-1 Storage Implementation
Stephen Burke, PPARC/RAL Jeff Templon, NIKHEF
WLCG Storage planning Andrea Sciabà
Data Management cluster summary
INFNGRID Workshop – Bari, Italy, October 2004
The LHCb Computing Data Challenge DC06
Summary of the dCache workshop
Presentation transcript:

Status of the SRM 2.2 MoU extension Flavia Donno Andrea Sciabà Grid Deployment Board April 8, 2009

Summary of items Protect spaces from generic users Restrict space release, staging, pool-to-pool copy VOMS awareness of SRM server The primary FQAN in proxy should determine authorization Space selection for read operations More control on pool usage Release files without knowing the ID of the request that staged them in srmLs should say in which spaces a file exists srmGetSpaceMetaData should return information about the used and available space in an SRM space Pass information to the SRM server to optimize tape usage Essential to have a high efficiency in staging

Space protection CASTOR Available, based on UID and GID dCache Protection against prestaging requests available by STEP09 DPM Spaces can be write protected. Multiple FQANs can be authorized to write (from 1.7.0). Prestaging: not applicable. StoRM Space ACLs as defined in MoU (v.1.4-RCx, x4). FQAN pattern matching supported as recommended by EGEE (EGEE-III-JRA1_FQAN_wildcard_v1.0.doc)

VOMS awareness CASTOR Not available. Timescale not yet defined dCache Available. ACLs for file system operations available before STEP09 DPM Available StoRM

Space selection for read operations CASTOR Available dCache Not available. Could implement pool selection using the TExtraInfo input parameter DPM Not applicable StoRM Not applicable (files have a unique copy). Will be available with tape integration

Release files without request ID CASTOR Supported, but srmReleaseFiles buggy (files still reported as pinned after), fixed in SRM 2.8 dCache Supported from 1.9.3 (in preparation) DPM Supported from 1.7.0 StoRM Supported

srmLs returns spaces CASTOR Not available dCache Not available (files can be in only one space token) DPM Available with 1.7.0 StoRM

srmGetSpaceMetadata CASTOR Available. It does not currently account for disk servers not in production. It will with CASTOR 2.1.8 dCache Available DPM StoRM

Client-driven tape optimization CASTOR Not client-driven. Can map namespace regions or disk pools to tape pools dCache Not client-driven. Can pass space token to HSM interconnect scripts, and can map namespace regions to tape families DPM Not applicable StoRM Not applicable now. For integration with tape backend, foreseen to have automatic optimization

Other news CASTOR dCache DPM StoRM All servers Ls will return checksum from SRM 2.8 with CASTOR 2.1.8 Initially it will not work with files created via GridFTP in “internal” mode (CERN); if created via RFIO or via GridFTP in “external” mode (Tier-1’s) it will work dCache Most effort on stability (asynchronous srmLs from 1.9.3) effort to support xrootd: to be evaluated Also working on NFS 4.1 and http support DPM srmCopy supported from 1.7.0 1.7.1 (to be certified) stores file checksum (http support already available) StoRM Ls returns Adler32 checksum (v.1.4) All servers Proper usage of SRM_INTERNAL_ERROR when server or backend is busy

Other news (II) Clients (GFAL/lcg-util, FTS) lcg-util FTS SRM workshop Will implement support for proper retry after an SRM_INTERNAL_ERROR by autumn Support for asynchronous srmLs in CVS for GFAL, will come with the first patch for FTS 2.2 lcg-util Correctly manages the case when multiple space tokens match a space description FTS Allows to specify space tokens, pin lifetime, file storage type and transfer parameters (to be certified) SRM workshop To be held on May 18-20 at DESY

Conclusions Not available for 2009/2010 data taking Client-driven tape optimization VOMS support in CASTOR Space selection in read operations in dCache Available for STEP09? File checksum in srmLs in CASTOR: yes, at CERN, with limitations. Yes, at Tier-1 sites, if they upgrade Prestaging and p2p copy protection in dCache: foreseen Release files without request ID in dCache: foreseen File ACLs in dCache: foreseen Space ACLs in StoRM: foreseen Question: what is still really needed? Are there new requirements for SRM?