1. SRM 2.2 Shaun de Witt (s.de.witt@rl.ac.uk)

2. Introduction Design Design Dependencies Dependencies Set-up Set-up Requirements Requirements Support Support Upcoming Upcoming

3. Thanks…. Core Development: Core Development: –Giuseppe Lo Presti, Tara Shah, Jiri Mencak Discussion, support, food, beer and wine Discussion, support, food, beer and wine –Olof Barring, Sebastien Ponce, and all CASTOR team (too many to mention), Jens Jensen SRM Developers SRM Developers –esp. Jean-Philippe Baud, Timur Perelmutov, Owen Synge SRM Testers SRM Testers – Alex Sim, Sophie Lemaitre, Gilbert Grosdidier, Flavia Donno

4. Design Database Centric Database Centric –Some re-use of CASTOR core components Thread library in daemon Thread library in daemon –Server uses simplified thread model DLF for both DLF for both –Request Flow depends on request type not all functions hit SRM database… not all functions hit SRM database… …but all of them hit the CASTOR database …but all of them hit the CASTOR database

5. Design Secure Secure –Uses GSI authentication –VOMS ready whatever that means whatever that means Server and Daemon multithreaded Server and Daemon multithreaded –configurable # threads –Throttled number of heavyweight threads srmLs and possibly srmChangeSpaceForFiles srmLs and possibly srmChangeSpaceForFiles

6. Design ServerDaemon SRM Database ClientsCASTOR

7. Data Flows Two types of request in SRM spec Two types of request in SRM spec Synchronous Synchronous –Client waits for response –Typically does not go through SRM database of daemon –Server talks directly to nameserver and/or stager Asynchronous Asynchronous –Request stored in SRM database. –Associated with srmStatusOfXXX call

8. DataFlows Two Examples Two Examples Directory Function (sync) Directory Function (sync) –e.g. srmMkdir, srmRmdir PUT function (async) PUT function (async) –srmPrepareToPut –srmStatusOfPutRequest –srmPutDone

9. Directory Functions Server Clients

10. Directory Functions Server ClientsCASTOR Nameserver

11. Directory Functions Server ClientsCASTOR Nameserver

12. Directory Functions Server ClientsCASTOR Nameserver

13. PUT operation (Prepare) ServerDaemon SRM Database ClientsCASTOR

14. PUT operation (Prepare) ServerDaemon SRM Database ClientsCASTOR

15. PUT operation (Prepare) ServerDaemon SRM Database ClientsCASTOR

16. PUT operation (Status) ServerDaemon SRM Database ClientsCASTOR

17. PUT operation (Status) ServerDaemon SRM Database ClientsCASTOR

18. PUT operation (Status) ServerDaemon SRM Database ClientsCASTOR

19. PUT operation (Status) ServerDaemon SRM Database ClientsCASTOR

20. PUT operation (Done) ServerDaemon SRM Database ClientsCASTOR

21. PUT operation (Done) ServerDaemon SRM Database ClientsCASTOR

22. PUT operation (Done) ServerDaemon SRM Database ClientsCASTOR

23. PUT operation (Done) ServerDaemon SRM Database ClientsCASTOR

24. Dependencies Since everything done using RPMs should be no trouble working then out Since everything done using RPMs should be no trouble working then out CASTOR: CASTOR: –2.1.1 or later CGSI-GSOAP plug-in: CGSI-GSOAP plug-in: –2.7-1.1.14 or later Globus Toolkit Globus Toolkit –vdt_globus_essentials-VDT1.2.2rh9_LCG-2 VOMS VOMS –glite-security-voms-api-c-1.6.16 Oracle Oracle –as per castor stager

25. Dependencies Server and Daemon in separate rpms Server and Daemon in separate rpms – allows east deployment on separate nodes Both have same dependencies Both have same dependencies –srmCopy is daemon side operation that needs security libraries

26. Set-Up Two configuration files Two configuration files –srm2.conf general configuration general configuration –srm2_storagemap.conf Maps storage areas (space tokens) to service class Maps storage areas (space tokens) to service class Oracle Oracle –Script to create tables –Need to set up tnsnames, ORASTAGERCONFIG –Need to pre-populate storageArea table

27. SRM2.conf NOTIFYHOST NOTIFYHOST –hostname where srmDaemon is running NOTIFYPORT NOTIFYPORT –port numbers which daemon is listening on one for GET/PUT one for GET/PUT one for COPY one for COPY PINTIME PINTIME –Garbage Collection Weighting –Not implemented by default in CASTOR

28. SRM2.conf MAXLSRETURNS/MAXLSCOUNT MAXLSRETURNS/MAXLSCOUNT –Max # returns from srmLs, and max allowable offset prevents too heavy load on srm prevents too heavy load on srm SERVERTHREADS/LSTHREADS SERVERTHREADS/LSTHREADS –SERVERTHREADS is max # of threads in srmServer –LSTHREADS is max # of Ls requests avoids accidental DNS attacks avoids accidental DNS attacks

29. SRM2.conf Stager mapping Stager mapping –SRM –SRM –default: SRMSTAGE_HOST SRMSTAGE_HOST For single castor instance sites For single castor instance sites

30. SRM2.conf SRM c2alice c2cms c2lhcb c2public vo=alice vo=cms vo=lhcb no vo info

31. srm2_storagemap.conf Maps SRM retention policy/access latency to space token Maps SRM retention policy/access latency to space token –hence to service class Format Format – TAPENDISKM – TAPENDISKM

32. srm2_storagemap.conf ALICE TAPE1DISK1 alimdc ALICE TAPE1DISK0 wan ALICE TAPE0DISK1 default ALICE default default DEFAULT TAPE1DISK1 dteam DEFAULT TAPE1DISK0 dteam DEFAULT TAPE0DISK1 dteam DEFAULT default dteam

33. srm2_storagemap.conf NOTE NOTE –Disk0 => Garbage Collection –Disk1 => No Garbage Collection –Tape1 => Migration –Tape0 => No Migration All should have default specified All should have default specified –Request with no space token or retentionPolicy info.

34. Oracle Setup Must initially set up service classes in CASTOR Must initially set up service classes in CASTOR –Service class name reused Populate StorageArea Table Populate StorageArea Table –by hand –VO production manager run srmReserveSpace, then update entry showing data available preferred since gets association between storage area and VO for free preferred since gets association between storage area and VO for free

35. Oracle Setup StorageArea table: StorageArea table: –spaceToken VARCHAR2(2048) –tokenDescription VARCHAR2(2048) –storageLifetime INTEGER –svcClass VARCHAR2(2048) –storageType NUMBER –id INTEGER PRIMARY KEY –srmUser INTEGER –parent INTEGER –storageStatus INTEGER

36. Setup ALICE example ALICE example Service Classes: Service Classes: –alimdc (Tape1Disk0) –recovery (Tape0Disk1) –wan (Tape1Disk0) –default (Tape1Disk0) –spare (Tape0Disk0)

37. srm2.conf #ALICE TAPE1DISK1 not supported ALICE TAPE1DISK0 alimdc ALICE TAPE0DISK1 recovery ALICE default default

38. StorageArea Table SpaceTokenTokenDesrcitionLifetimeSvcClassStorageTypeStorageStatus <uuid>alice::alimdcalimdc02 <uuid>alice::recoveryrecovery02 <uuid>alice::wanwan02 <uuid>alice::defaultdefault02 <uuid>alice::sparespare24

39. Special Values Lifetime -1 => infinite (user controlled) Lifetime -1 => infinite (user controlled) –lifetime of space, not files in space StorageType StorageType –0, 1, 2 –permanent, durable. volatile StorageStatus StorageStatus –0 = REQUESTED –1 = ALLOCATED –2 = IN USE –3 = DEALLOCATED –4 = EXPIRED

40. Requirements Suitable database and table space Suitable database and table space –Current at RAL and CERN SRM uses same database instance as stager –Unclear of load, may need its own oracle instance Suitable port open Suitable port open –8443 is normally used

41. Support Supported by Giuseppe Lo Presti (CERN) and myself (RAL) Supported by Giuseppe Lo Presti (CERN) and myself (RAL) –Details to be worked out –Informal e-mails should get reasonable response but need to integrate into tracking s/w at CERN or RAL? but need to integrate into tracking s/w at CERN or RAL? –srm mailing list might be resurrected for support On-site support for deployment at other sites to be discussed On-site support for deployment at other sites to be discussed Client tools being tested now. Client tools being tested now.

42. Upcoming ChangeSpaceForFiles ChangeSpaceForFiles –Could be done wholly through SRM but easier internally to CASTOR UpdateSpace UpdateSpace –Needs reimplementing similar to ReserveSpace Lots and lots of bug fixes Lots and lots of bug fixes –Finally getting lots of testers! castor-gridftp2 support already in castor-gridftp2 support already in