A. Sim, CRD, L B N L 1 OSG Site Administrators Meeting, Dec. 13, 2007 Berkeley Storage Manager (BeStMan) Alex Sim Scientific Data Management Research Group Computational Research Division Lawrence Berkeley National Laboratory

A. Sim, CRD, L B N L 2 OSG Site Administrators Meeting, Dec. 13, 2007 Berkeley Storage Manager (BeStMan) Current version (Dec. 11, 2007)Current version (Dec. 11, 2007) Full implementation of SRM v2.2Full implementation of SRM v2.2 Following WLCG SRM MoU and more No support on ACL related functions such as srmSetPermission For small disk-based storage and file systemsFor small disk-based storage and file systems NFS, PVFS, AFS, NGFS, Lustre, XrootdFS Mass Storage supportMass Storage support Currently, for HPSS and NCAR-MSS Plug-in capability for easy customization for underlying storage E.g. SRM/L-Store, SRM/XrootD Use of existing file transfer servicesUse of existing file transfer services GSIFTP, FTP, BBFTP, HTTP, HTTPS

A. Sim, CRD, L B N L 3 OSG Site Administrators Meeting, Dec. 13, 2007 Who is it for? Smaller sites with shared and limited resourcesSmaller sites with shared and limited resources Sites where minimal administrative effort is neededSites where minimal administrative effort is needed Main use is for disks with unix-based file systemsMain use is for disks with unix-based file systems In OSG, small tier2 sites and tier3 sitesIn OSG, small tier2 sites and tier3 sites Sites/users that need dynamic invocation of an SRMSites/users that need dynamic invocation of an SRM Can be customized for different file system and MSSCan be customized for different file system and MSS Sites that need great supportSites that need great support

A. Sim, CRD, L B N L 4 OSG Site Administrators Meeting, Dec. 13, 2007 Berkeley Storage Manager (BeStMan) Java implementationJava implementation Designed to work with unix- based disk systemsDesigned to work with unix- based disk systems As well as MSS to stage/archive from/to its own diskAs well as MSS to stage/archive from/to its own disk Adaptable to other file systems and storages (e.g. NCAR MSS, VU L-Store, TTU Lustre, SLAC XrootdFS)Adaptable to other file systems and storages (e.g. NCAR MSS, VU L-Store, TTU Lustre, SLAC XrootdFS) Uses in-memory database (BerkeleyDB)Uses in-memory database (BerkeleyDB) Multiple transfer protocol supportMultiple transfer protocol support Space reservationSpace reservation Directory managementDirectory management Can copy files from/to remote SRMs/GridFTP serversCan copy files from/to remote SRMs/GridFTP servers Can copy entire directory recursivelyCan copy entire directory recursively Large scale data movement of thousands of files Recovers from transient failures (e.g. MSS maintenance, network down) Local PolicyLocal Policy Fair request processing File placement in disk Garbage collection

A. Sim, CRD, L B N L 5 OSG Site Administrators Meeting, Dec. 13, 2007 BeStMan Features Multiple disk partition supportMultiple disk partition support Default space management for files with lifetimeDefault space management for files with lifetime Allocation of space, garbage collectionAllocation of space, garbage collection Dynamic space reservationDynamic space reservation Support for multiple file transfer serversSupport for multiple file transfer servers E.g. configure for 5 GridFTP serversE.g. configure for 5 GridFTP servers Per-user (not per-request) request managementPer-user (not per-request) request management Multi-file/directory per requestMulti-file/directory per request Incoming and outgoing file transfer queue management and transfer monitoringIncoming and outgoing file transfer queue management and transfer monitoring Support file sharing and file streamingSupport file sharing and file streaming Easy adaptability to Unix-based file systemsEasy adaptability to Unix-based file systems Easy adaptability and integration to special project environmentsEasy adaptability and integration to special project environments API support for customization for local mass storage systemsAPI support for customization for local mass storage systems Simple installation and easy maintenanceSimple installation and easy maintenance

A. Sim, CRD, L B N L 6 OSG Site Administrators Meeting, Dec. 13, 2007 A few hints on what BeStMan can do If the target host does not have an SRM, BeStMan can be installed, configured, started and stopped for user requests through a normal job submission.If the target host does not have an SRM, BeStMan can be installed, configured, started and stopped for user requests through a normal job submission. Configurable support for accessibility to user’s own disk path as source url or target url in the same SURL format.Configurable support for accessibility to user’s own disk path as source url or target url in the same SURL format. User’s own file path srm://host:port/srm/v2/server?SFN=/home/john/myfile SRM cache srm://host:port/srm/v2/server?SFN=/srmcache/john/myfile2 Works with FTS file copyWorks with FTS file copy Between BeStMan and other SRM v2.2 servers Between BeStMan and SRM v1.1 servers

A. Sim, CRD, L B N L 7 OSG Site Administrators Meeting, Dec. 13, 2007 SRB (iRODS) SDSC SINICA LBNL EGEE Interoperability with other SRM v2.2 Client User/application CASTOR Disk BeStMan xrootd BNL SLAC LBNL dCache DPM mySQL DB

A. Sim, CRD, L B N L 8 OSG Site Administrators Meeting, Dec. 13, 2007 Some Use Cases STAR experimentSTAR experiment Data replication between BNL and LBNL HPSS access at BNL and NERSC SRMs in production for over 4 years Part of analysis scenario to move job-generated data files from PDSF/NERSC to remote BNL storage Earth System GridEarth System Grid Serving about 6000 users Over a million files and 170TB of climate data from 5 storage sites with SRMs (LANL, LLNL, NCAR, NERSC, ORNL) Uses an adapted SRM for NCAR’s own MSS

A. Sim, CRD, L B N L 9 OSG Site Administrators Meeting, Dec. 13, 2007Summary BeStMan is an implementation of SRM v2.2.BeStMan is an implementation of SRM v2.2. Current version (as of Dec. 11, 2007) Interoperates with other SRM v2.2 implementationsInteroperates with other SRM v2.2 implementations Open Science Grid WLCG/EGEE Simple installation and easy maintenanceSimple installation and easy maintenance

A. Sim, CRD, L B N L 10 OSG Site Administrators Meeting, Dec. 13, 2007 Documents and Support BeStManBeStMan Downloadable through VDT SRM Collaboration and SRM SpecificationsSRM Collaboration and SRM Specifications Contact and support : and support :

A. Sim, CRD, L B N L 11 OSG Site Administrators Meeting, Dec. 13, 2007 BeStMan Installation Demo

A. Sim, CRD, L B N L 12 OSG Site Administrators Meeting, Dec. 13, 2007Preparation Grid service certificateGrid service certificate GridFTP server hostname, port numberGridFTP server hostname, port number Default: $hostname:2811 GLOBUS_TCP_PORT_RANGEGLOBUS_TCP_PORT_RANGE If $hostname has a firewall Two open port numbers to be assigned to BeStManTwo open port numbers to be assigned to BeStMan Default: 8080, 8443 Local disk path and size information to be managed by BeStManLocal disk path and size information to be managed by BeStMan Log file path informationLog file path information Default: /var/log Java 1.5.0_x installation pathJava 1.5.0_x installation path Default from $PATH BeStMan installation directoryBeStMan installation directory Default: current directory BeStMan downloads from downloads from Or VDT pacman installation

A. Sim, CRD, L B N L 13 OSG Site Administrators Meeting, Dec. 13, 2007 Preparation : example Grid service certificateGrid service certificate /DC=org/DC=doegrids/OU=Services/CN=srmdemo/dmx09.lbl.gov $HOME/osgdemo/srmdemocert.pem and $HOME/osgdemo/srmdemokey.pem GridFTP server hostname, port numberGridFTP server hostname, port number dmx09.lbl.gov GLOBUS_TCP_PORT_RANGEGLOBUS_TCP_PORT_RANGE 6201,6299 Two open port numbers to be assigned to BeStManTwo open port numbers to be assigned to BeStMan 6249, 6250 Local disk path and size information to be managedLocal disk path and size information to be managed /data/osg/demo/data : 20000MB Log file path informationLog file path information /data/osg/demo/log Java 1.5.0_x installation pathJava 1.5.0_x installation path /software/jdk1.5.0_12 BeStMan installation directoryBeStMan installation directory /data/osg/bestman

A. Sim, CRD, L B N L 14 OSG Site Administrators Meeting, Dec. 13, 2007Configuration./configure \./configure \ --with-globus-tcp-port-range=6201,6299 \ --with-http-port=6249 \ --with-https-port=6250 \ --with-eventlog-path=/data/osg/demo/log \ --with-cachelog-path=/data/osg/demo/log \ --with-replica-storage-path=/data/osg/demo/data \ --with-replica-storage-size=20000 \ --with-certfile-path=/home/users/asim/osgdemo/srmcert.pem \ --with-keyfile-path=/home/users/asim/osgdemo/srmkey.pem

A. Sim, CRD, L B N L 15 OSG Site Administrators Meeting, Dec. 13, 2007 BeStMan server start/stop As a user/service loginAs a user/service login $SRM_HOME/sbin/SXXbestman.personal start $SRM_HOME/sbin/bestman.server >& /tmp/bestman-.log 2>&1 & $SRM_HOME/sbin/bestman.server $SRM_HOME/sbin/SXXbestman.personal stop As a root loginAs a root login $SRM_HOME/sbin/SXXbestman start su $SRMOWNER -c "$SRM_HOME/sbin/bestman.server &“ $SRM_HOME/sbin/ SXXbestman stop

A. Sim, CRD, L B N L 16 OSG Site Administrators Meeting, Dec. 13, 2007 Sample client runs (1) setenv SRM_HOME /data/osg/bestmansetenv SRM_HOME /data/osg/bestman Ping:Ping: $SRM_HOME/bin/srm-ping srm://dmx09.lbl.gov:6250/srm/v2/server Put:Put: $SRM_HOME/bin/srm-copy file:////tmp/test.data srm://dmx09.lbl.gov:6250/srm/v2/server\?SFN=~/osg.test.data Ls:Ls: $SRM_HOME/bin/srm-ls srm://dmx09.lbl.gov:6250/srm/v2/server\?SFN=~/osg.test.data Get:Get: $SRM_HOME/bin/srm-copy srm://dmx09.lbl.gov:6250/srm/v2/server\?SFN=~/osg.test.data file:////tmp/osg.data Rm:Rm: $SRM_HOME/bin/srm-rm srm://dmx09.lbl.gov:6250/srm/v2/server\?SFN=~/osg.test.data Mkdir:Mkdir: $SRM_HOME/bin/srm-mkdir srm://dmx09.lbl.gov:6250/srm/v2/server\?SFN=~/osg.test.dir Rmdir:Rmdir: $SRM_HOME/bin/srm-rmdir srm://dmx09.lbl.gov:6250/srm/v2/server\?SFN=~/osg.test.dir

A. Sim, CRD, L B N L 17 OSG Site Administrators Meeting, Dec. 13, 2007 Sample client runs (2) space reservationspace reservation $SRM_HOME/bin/srm-sp-reserve -serviceurl srm://dmx09.lbl.gov:6250/srm/v2/server \ -size gsize 900 -lifetime 900 Or, $SRM_HOME/bin/srm-space -reserve -serviceurl srm://dmx09.lbl.gov:6250/srm/v2/server \ -size gsize 900 -lifetime 900 space updatespace update $SRM_HOME/bin/srm-sp-update -serviceurl srm://dmx09.lbl.gov:6250/srm/v2/server \ -size gsize lifetime 300 -spacetoken V.10 Or, $SRM_HOME/bin/srm-space -serviceurl srm://dmx09.lbl.gov:6250/srm/v2/server \ -size gsize lifetime 300 -update V.10 space tokens retrievalspace tokens retrieval $SRM_HOME/bin/srm-sp-tokens -serviceurl srm://dmx09.lbl.gov:6250/srm/v2/server Or, $SRM_HOME/bin/srm-space -serviceurl srm://dmx09.lbl.gov:6250/srm/v2/server -getspacetoken space information retrievalspace information retrieval $SRM_HOME/bin/srm-sp-info -serviceurl srm://dmx09.lbl.gov:6250/srm/v2/server -spacetoken V.10 Or, $SRM_HOME/bin/srm-space -serviceurl srm://dmx09.lbl.gov:6250/srm/v2/server -getspacetokenmeta V.10

A. Sim, CRD, L B N L 18 OSG Site Administrators Meeting, Dec. 13, 2007 Sample client runs (3) put operation with the reserved spaceput operation with the reserved space $SRM_HOME/bin/srm-copy file:////tmp/test.data srm://dmx09.lbl.gov:6250/srm/v2/server\?SFN=~/osg.test2.data -spacetoken V.10 purge an SURL in the reserved spacepurge an SURL in the reserved space $SRM_HOME/bin/srm-sp-purge -serviceurl srm://dmx09.lbl.gov:6250/srm/v2/server \ -spacetoken V.10 -s "srm://dmx09.lbl.gov:6250/srm/v2/server?SFN=~/osg.test2.data" Or, $SRM_HOME/bin/srm-space -serviceurl srm://dmx09.lbl.gov:6250/srm/v2/server \ -s "srm://dmx09.lbl.gov:6250/srm/v2/server?SFN=~/osg.test2.data“ -purge V.10 space releasespace release $SRM_HOME/bin/srm-sp-release -serviceurl srm://dmx09.lbl.gov:6250/srm/v2/server \ -spacetoken V.10 $SRM_HOME/bin/srm-space -serviceurl srm://dmx09.lbl.gov:6250/srm/v2/server \ -release V.10

A. Sim, CRD, L B N L 19 OSG Site Administrators Meeting, Dec. 13, 2007 Extra Slides

A. Sim, CRD, L B N L 20 OSG Site Administrators Meeting, Dec. 13, 2007 Data Replication in STAR srmCopy (files/directories) GET (one file at a time) GridFTP GET (pull mode) stage files archive files Network transfer Disk Cache Anywhere SRM-COPY client command SRM/BeStMan LBNL Disk Cache SRM/BeStMan BNL Recovers from file transfer failures Recovers from staging failures Recovers from archiving failures Make equivalent Directory BROWSE RELEASE Allocate space on disk (performs reads) (performs writes)

A. Sim, CRD, L B N L 21 OSG Site Administrators Meeting, Dec. 13, 2007 STAR Analysis scenario (1) BeStMan Disk Cache DISK CACHE Client Job Gate Node Worker Nodes Disk Client Job Disk Cache BeStMan Disk Cache Disk BeStMan Disk GridFTP server Disk SRMs Client Job submission Remote sites A site

A. Sim, CRD, L B N L 22 OSG Site Administrators Meeting, Dec. 13, 2007 STAR Analysis scenario (2) BeStMan Disk Cache DISK CACHE Client Job Gate Node Worker Nodes Disk Client Job Disk Cache BeStMan Disk Cache Disk BeStMan Disk GridFTP server Disk SRMs Client Client Job submission SRM Job submission A site Remote sites

A. Sim, CRD, L B N L 23 OSG Site Administrators Meeting, Dec. 13, 2007 SRM works in concert with other Grid components in ESG SRM works in concert with other Grid components in ESG MCS Metadata Cataloguing Services RLS Replica Location Services MyProxy MSS Mass Torage System DISK HPSS DISK HPSS DRM Storage Resource Management DRM Storage Resource Management HRM Storage Resource Management HRM Storage Resource Management HRM Storage Resource Management HRM Storage Resource Management HRM Storage Resource Management HRM Storage Resource Management GridFTP server GridFTP server GridFTP server GridFTP server GridFTP server GridFTP server GridFTP server OPeNDAP-g LBNL LLNL ISI NCAR ORNL ANL DRM Storage Resource Management DRM Storage Resource Management GridFTP server GridFTP server LANL GridFTP service RLS Globus Security infrastructure IPCC Portal ESG Metadata DB User DB XML data catalogs ESG CA LAHFS RLS XML data catalogs FTP server ESG Portal Monitoring Discovery ervices DISK