A. Sim, CRD, L B N L 1 Oct. 23, 2008 BeStMan Extra Slides

A. Sim, CRD, L B N L 2 Oct. 23, 2008 BeStMan Features Multiple disk partition supportMultiple disk partition support Default space management for files with lifetimeDefault space management for files with lifetime Allocation of space, garbage collectionAllocation of space, garbage collection Dynamic space reservationDynamic space reservation Support for multiple file transfer serversSupport for multiple file transfer servers E.g. configure for 5 GridFTP serversE.g. configure for 5 GridFTP servers Per-user (not per-request) request managementPer-user (not per-request) request management Multi-file/directory per requestMulti-file/directory per request Incoming and outgoing file transfer queue management and transfer monitoringIncoming and outgoing file transfer queue management and transfer monitoring Support file sharing and file streamingSupport file sharing and file streaming Easy adaptability to Unix-based file systemsEasy adaptability to Unix-based file systems Easy adaptability and integration to special project environmentsEasy adaptability and integration to special project environments API support for customization for local mass storage systemsAPI support for customization for local mass storage systems Simple installation and easy maintenanceSimple installation and easy maintenance

A. Sim, CRD, L B N L 3 Oct. 23, 2008 Berkeley Storage Manager (BeStMan) Java implementationJava implementation Designed to work with Unix-based disk file systemsDesigned to work with Unix-based disk file systems As well as MSS to stage/archive from/to its own diskAs well as MSS to stage/archive from/to its own disk Adaptable to other special file systems and storages (e.g. NCAR MSS, SLAC XrootdFS)Adaptable to other special file systems and storages (e.g. NCAR MSS, SLAC XrootdFS) Uses in-memory database (BerkeleyDB)Uses in-memory database (BerkeleyDB) Multiple transfer protocol supportMultiple transfer protocol support Space reservationSpace reservation Directory managementDirectory management Can copy files from/to remote SRMs/GridFTP serversCan copy files from/to remote SRMs/GridFTP servers Can copy entire directory recursivelyCan copy entire directory recursively Large scale data movement of thousands of files Recovers from transient failures (e.g. MSS maintenance, network down) Local PolicyLocal Policy Fair request processing File placement in disk Garbage collection

A. Sim, CRD, L B N L 4 Oct. 23, 2008 Some Use Cases STAR experimentSTAR experiment Data replication between BNL and LBNL HPSS access at BNL and NERSC SRMs in production for over 4 years Part of analysis scenario to move job-generated data files from PDSF/NERSC to remote BNL storage Earth System GridEarth System Grid Serving about 6000 users Over a million files and 170TB of climate data from 5 storage sites with SRMs (LANL, LLNL, NCAR, NERSC, ORNL) Uses an adapted SRM for NCAR’s own MSS

A. Sim, CRD, L B N L 5 Oct. 23, 2008 Data Replication from BNL to LBNLData Replication from BNL to LBNL SRMs in production for over 4 years Event processing in Grid CollectorEvent processing in Grid Collector Prototype uses SRMs and FastBit indexing embedded in STAR framework STAR analysis frameworkSTAR analysis framework Job driven data movement 1.Use BeStMan to bring files into local disk from a remote file repository 2.Execute jobs that access “staged in” files in local disk 3.Job creates an output file on local disk 4.Job uses BeStMan to move the output file from local storage to remote archival location 5.SRM cleans up local disk when transfer complete 6.Can use any other SRMs implementing v2.2 STAR experiment

A. Sim, CRD, L B N L 6 Oct. 23, 2008 STAR Analysis scenario BeStMan Disk Cache DISK CACHE Client Job Gate Node Worker Nodes Disk Client Job Disk Cache BeStMan Disk Cache Disk BeStMan Disk GridFTP server Disk SRMs Client Job submission Remote sites A site

A. Sim, CRD, L B N L 7 Oct. 23, 2008 BeStMan Gateway Mode Extra Slides

A. Sim, CRD, L B N L 8 Oct. 23, 2008 How it works all together in Put/Get File System Gridftp server BeStMan Gateway Client srmPrepareToGet/Put TURL GridFTP file transfers srmReleaseFiles/srmPutDone

A. Sim, CRD, L B N L 9 Oct. 23, 2008 How it works all together in Ls/Rm/Mkdir/Rmdir File system Gridftp server BeStMan Gateway Client srmLs/srmRm/srmMkdir/srmRmdir

A. Sim, CRD, L B N L 10 Oct. 23, 2008 Extra Slides on SRM

A. Sim, CRD, L B N L 11 Oct. 23, 2008 Storage Resource Managers (SRMs) are middleware components Storage Resource Managers (SRMs) are middleware components whose function is to provide dynamic space allocation and file management on shared storage components on the Grid Different implementations for underlying storage systems based on the SRM specification What is SRM?

A. Sim, CRD, L B N L 12 Oct. 23, 2008 Get/put files from/into spacesGet/put files from/into spaces archived files on mass storage systems Shared storage space allocation & reservationShared storage space allocation & reservation important for data intensive applications File transfers from/to remote sites, file replicationFile transfers from/to remote sites, file replication Negotiate transfer protocolsNegotiate transfer protocols File and space management with lifetimeFile and space management with lifetime support non-blocking (asynchronous) requestssupport non-blocking (asynchronous) requests Directory managementDirectory management Interoperate with other SRMsInteroperate with other SRMs SRMs in the data grid

A. Sim, CRD, L B N L 13 Oct. 23, 2008History 7 year of Storage Resource (SRM) Management activity7 year of Storage Resource (SRM) Management activity Experience with system implementations v.1.x Experience with system implementations v.1.x MSS: HPSS (LBNL, ORNL, BNL), Enstore (Fermi), JasMINE (Jlab), Castor (CERN), MSS (NCAR), SE (RAL) … Disk systems: DRM(LBNL), dCache(Fermi), DPM(CERN), jSRM (Jlab), … SRM v2.1 spec was finalized – 2003SRM v2.1 spec was finalized – 2003 GSM: BOF at GGF8 - June 2003GSM: BOF at GGF8 - June 2003 SRM v2.2 spec was finalized – May 2006SRM v2.2 spec was finalized – May 2006 SRM v3.0 spec being discussed SRM v3.0 spec being discussed

A. Sim, CRD, L B N L 14 Oct. 23, 2008 SRM v2.2 Interface Data transfer functions to get files into SRM spaces from the client's local system or from other remote storage systems, and to retrieve themData transfer functions to get files into SRM spaces from the client's local system or from other remote storage systems, and to retrieve them srmPrepareToGet, srmPrepareToPut, srmBringOnline, srmCopy Space management functions to reserve, release, and manage spaces, their types and lifetimes.Space management functions to reserve, release, and manage spaces, their types and lifetimes. srmReserveSpace, srmReleaseSpace, srmUpdateSpace, srmGetSpaceTokens Lifetime management functions to manage lifetimes of space and files.Lifetime management functions to manage lifetimes of space and files. srmReleaseFiles, srmPutDone, srmExtendFileLifeTime Directory management functions to create/remove directories, rename files, remove files and retrieve file information.Directory management functions to create/remove directories, rename files, remove files and retrieve file information. srmMkdir, srmRmdir, srmMv, srmRm, srmLs Request management functions to query status of requests and manage requestsRequest management functions to query status of requests and manage requests srmStatusOf{Get,Put,Copy,BringOnline}Request, srmGetRequestSummary, srmGetRequestTokens, srmAbortRequest, srmAbortFiles, srmSuspendRequest, srmResumeRequest Other functions include Discovery and Permission functionsOther functions include Discovery and Permission functions srmPing, srmGetTransferProtocols, srmCheckPermission, srmSetPermission, etc.

A. Sim, CRD, L B N L 15 Oct. 23, 2008 Interoperability ? Storage Resource Managers (SRMs) are based on a common interface specification.Storage Resource Managers (SRMs) are based on a common interface specification. SRMs can have different implementations for the underlying storage systems. Compatibility and interoperability need to be tested according to the specification. 5 server implementations are currently available for v2.25 server implementations are currently available for v2.2 CASTOR (CERN, RAL) dCache (FNAL, DESY) DPM (CERN) StoRM (Italy) BeStMan (LBNL)