SRM & SE Jens G Jensen WP5 ATF, December 2002

Collaborators Rutherford Appleton (ATLAS datastore) CERN (CASTOR) Fermilab Jefferson Lab Lawrence Berkeley –Americans have: ENSTORE, SAM, … Also present: WP2 For an overview of storage systems see D5.1

File pinning & replication Local SRM Remote SRM RC file (replica) Client fetches a file using srmGet(LFN,SURL) Local SRM gets a replica identified by LFN. Issue: LFN is optional Other issues: What are defautl ACLs on replicas? Does the local SRM have to check ACLs from the remote SRM?

RCs SRM must be able to interoperate with several different RCs Getting SURLs (RC  SRM) is easy, but: –Does SRM need reverse lookup (SURL  LFN)? –How to notify RC that a file has been deleted? Add hooks to SRM?

SRM file lifetime overview Files are: Volatile (e.g. pinned files) –Files get deleted by SE when they time out –Files created as volatile are never written to tape Durable (e.g. permanent but not in final location) –Files can be deleted only by administrator or owner –Files still have a lifetime – owner and/or admin notified when lifetime expires Permanent (e.g. your important experiment) –Files can be deleted only by administrator or owner

New lifetime stuff: space Each user has space with a per-user (probably) quota for each type of space Users must reserve space before they can reserve for writing or reading Users are charged for the type and amount of space they reserve Space can be –Volatile – reservations not guaranteed beyond a minimum –Durable – notify user if file didn’t move, space wasn’t released, etc – if user doesn’t react, notify admin –Permanent SRM Proposal: space lifetime is encoded in the filepath!!! (JLAB has this) Proposal: space has a lifetime as well.

Storage Semantics VolatileDurablePermanent

copyToSpace Users can “give away” files in their own space to another user –First step: the user who owns the files gives them away –Second step: the other user takes over the files –Done without copying in the SRM, but quotas are adjusted accordingly Alternatively, let users “give” files to other users without deleting them from their own space –copy-on-write

SRM possibly wildly inaccurate status Get PULL Put PUSH Copy File Lifetime Space Lifetime GSIFTP SE SE 1.0 TB 2.0 SE 1.0 TB 2.0 SE 1.1 TB 2.1 SE 1.1 TB 2.1 SE 1.x x>=1 SE 1.0 TB 2.0 FermiOK only vol --OK JLABOK --OK

Comparing SRM v2.{0,1} to SE 1.0 Get, Put: supported –Get in PULL mode (same as SRM) –Put in PUSH mode (same as SRM) File lifetime: will be easy to support –Currently (pre-1.0) all files are permanent… Space management – currently supported as volatile, best-effort only (i.e. very little support) Information providers: supported soon Directory stuff : mostly supported (not rmdir, no recursion) Support for arbitrarily long LFNs: OK Need to have API for more detailed comparison…

Need for POSIX style access RFIO (CERN, and other sites with HPSS) DCAP (FermiLab) Globus XIO? file: Pick one, and call it a standard?

Security Americans expect to use CAS –Possibly need for interim solution until CAS is sufficiently mature? Need for “weak” ftp? –Users “delegate” username/password to SRM Europeans expect to use VOMS –SE solution until VOMS is sufficiently mature: GACL (SE will always support GACL) –SE can use CAS as well but support not planned Use GSI-HTTPS for now – common ground essential for interoperability Question: are CAS and VOMS interchangeable?

ACLs How to manage ACLs on replicas? –Replicas are read-only; or –Replicas are writable by whoever created the replica –SRM may have to contact the SRM containing the master to pick up changes to the ACL SRM proposal: default is world readable for volatile space, owner access only for permanent SE proposal –Have default ACL per user and/or per directory

Web services Fermi uses GLUE but plans to switch to Apache/Tomcat/Axis. “Secret” SRM v1.0 WSDL available… New WSDL spec for v2.1 specification in preparation (Fermi) SOAP interoperability issues, particularly with parameters more complicated than strings and ints – e.g. (WSDL) structs. Need to be OGSI compliant…

Schema Agree that GLUE is important More effort needed?

How to ensure that users behave? Good behaviour == close file && release pin Make them pay… Charge per space reservations and/or size of files in user’s reservation? –CASTOR charges per MB on tape Count number of expired pins (in durable and permanent only!?) What to do with files that are opened and never closed? –Can be detected sometimes (local files, NFS?, /grid?) –CASTOR currently does nothing

Collaborations… Agree standard protocols, accept each others certificates, etc.  Testing Interoperability Sharing code is seen as less important –Occasional licence conflict –People would rather have more independent implementations