Computing Sciences Directorate, L B N L 1 CHEP 2003 Storage Resource Management In the Grid Environment Alex Sim Junmin Gu Arie Shoshani Scientific Data Management Group Lawrence Berkeley National Laboratory

Computing Sciences Directorate, L B N L 2 CHEP 2003 Outline What are Storage Resource Managers - MotivationWhat are Storage Resource Managers - Motivation General Analysis Scenario and the use of SRMsGeneral Analysis Scenario and the use of SRMs SRM functionalitySRM functionality Real examples of working SRMsReal examples of working SRMs Advantages of using SRMsAdvantages of using SRMs Conclusions and Future WorkConclusions and Future Work

Computing Sciences Directorate, L B N L 3 CHEP 2003Motivation Grid architecture needs to include reservation & scheduling of:Grid architecture needs to include reservation & scheduling of: Compute resources Storage resources Network resources Storage Resource Managers (SRMs) role in the data grid architectureStorage Resource Managers (SRMs) role in the data grid architecture Shared storage resource allocation & scheduling Especially important for data intensive applications Often files are archived on a mass storage system (MSS) Wide area networks – minimize transfers large scientific collaborations (100’s of nodes, 1000’s of clients) – opportunities for file sharing File replication and caching may be used Need to support non-blocking (asynchronous) requests

Computing Sciences Directorate, L B N L 4 CHEP 2003 General Analysis Scenario MSS Request Executer Storage Resource Manager Metadata catalog Replica catalog Network Weather Service logical query network client... Request Interpreter request planning A set of logical files Execution plan and site-specific files Client’s site... Disk Cache Disk Cache Compute Engine Disk Cache Compute Resource Manager Storage Resource Manager Compute Engine Disk Cache Requests for data placement and remote computation Site 2 Site 1 Site N Storage Resource Manager Storage Resource Manager Compute Resource Manager result files Execution DAG

Computing Sciences Directorate, L B N L 5 CHEP 2003 SRM is a Service SRM functionalitySRM functionality Manage space Negotiate and assign space to users Manage “lifetime” of spaces Manage files on behalf of a user Pin files in storage till they are released Manage “lifetime” of files Manage action when pins expire (depends on file types) Manage file sharing Policies on what should reside on a storage resource at any one time Policies on what to evict when space is needed Get files from remote locations when necessary Purpose: to simplify client’s task Manage multi-file requests A brokering function: queue file requests, pre-stage when possible Provide grid access to/from mass storage systems HPSS (LBNL, ORNL, BNL), Enstore (Fermi), JasMINE (Jlab), Castor (CERN), MSS (NCAR), …

Computing Sciences Directorate, L B N L 6 CHEP 2003 Types of SRMs Types of storage resource managersTypes of storage resource managers Disk Resource Manager (DRM) Manages one or more disk resources Tape Resource Manager (TRM) Manages access to a tertiary storage system (e.g. HPSS) Hierarchical Resource Manager (HRM=TRM + DRM) An SRM that stages files from tertiary storage into its disk cache SRMs and File transfersSRMs and File transfers SRMs DO NOT perform file transfer SRMs DO invoke file transfer service if needed (GridFTP, FTP, HTTP, …) SRMs DO monitor transfers and recover from failures TRM: from/to MSS DRM: from/to network

Computing Sciences Directorate, L B N L 7 CHEP 2003 Analysis Scenario for Local Computation Analysis Scenario for Local Computation Uniform SRM interface tape system HRM Request Executer DRM Disk Cache Metadata catalog Replica catalog Network Weather Service logical query pinning & file transfer requests network DRM Disk Cache client... Request Interpreter request planning logical files site-specific files Client’s site... Disk Cache site-specific files requests Compute Engine

Computing Sciences Directorate, L B N L 8 CHEP 2003 SRM works with disk caches as well as legacy systems DRM Disk Cache Disk Cache Disk Cache Disk Cache Berkeley ChicagoLivermore HRMGridFTP FTP Disk Cache BIT-MAP Index Request Manager File Transfer Monitoring DRM GridFTP Denver client server Logical Request Data Path Control path Legend: SC 2001 Demo For HENP – STAR Experiment

Computing Sciences Directorate, L B N L 9 CHEP 2003 Tomcat servlet engine Tomcat servlet engine MCS Metadata Cataloguing Services MCS Metadata Cataloguing Services RLS Replica Location Services RLS Replica Location Services SOAP RMI MyProxy server MyProxy server MCS client RLS client MyProxy client GRAM gatekeeper GRAM gatekeeper CAS Community Authorization Services CAS Community Authorization Services CAS client NCAR-MSS Mass Storage System HPSS High Performance Storage System HPSS High Performance Storage System DRM Storage Resource Management DRM Storage Resource Management HRM Storage Resource Management HRM Storage Resource Management HRM Storage Resource Management HRM Storage Resource Management HRM Storage Resource Management HRM Storage Resource Management gridFTP server gridFTP server gridFTP server gridFTP server gridFTP server gridFTP server gridFTP server gridFTP server openDAPg server openDAPg server gridFTP Striped server gridFTP Striped server LBNL LLNL USC-ISI NCAR ORNL ANL DRM Storage Resource Management DRM Storage Resource Management disk Earth Science Grid Demo - SC 2002

Computing Sciences Directorate, L B N L 10 CHEP 2003 Uniformity of Interface  Compatibility of SRMs SRM Enstore JASMine Client USER/APPLICATIONS Grid Middleware SRM DCache SRM CASTOR SRM Disk Cache

Computing Sciences Directorate, L B N L 11 CHEP 2003 High Level View of SRM setup in SC 2002 SRM Enstore JASMine Client (USER/APPLICATIONS)

Computing Sciences Directorate, L B N L 12 CHEP 2003 Screen Dump of Demo at Fermi Booth

Computing Sciences Directorate, L B N L 13 CHEP 2003 Where do SRMs belong in the Grid architecture?

Computing Sciences Directorate, L B N L 14 CHEP 2003 SRMs provide a brokering service by supporting multi-file requests

Computing Sciences Directorate, L B N L 15 CHEP 2003 SRMs use in STAR for Robust Muti-file replication Anywhere BNL Disk Cache Disk Cache HRM-COPY (thousands of files) SRM-GET (one file at a time) HRM-Client Command-line Interface HRM (performs writes) HRM (performs reads) LBNL GridFTP GET (pull mode) stage files archive files Network transfer Get list of files Recovers from staging failures Recovers from file transfer failures Recovers from archiving failures

Computing Sciences Directorate, L B N L 16 CHEP 2003 Web-Based File Monitoring Tool Shows: -Files already transferred - Files during transfer - Files to be transferred Also shows for each file: -Source URL -Target URL -Transfer rate

Computing Sciences Directorate, L B N L 17 CHEP 2003 GridFTP-HPSS Access Provided through HRM HRM GridFTP SRM-API GridFTP-API Client HRM GridFTP move SRM-API GridFTP-API Client Using HRM protocol New: GridFTP-HPSS through HRM GridFTP entry No modifications to the MSS Managing queues of multiple requests to the MSS Minimizing tape mounts Recovers from MSS transient failures

Computing Sciences Directorate, L B N L 18 CHEP 2003 GridFTP-HRM-Layer implementation detail HRM GridFTP-API Client GridFTP entry GridFTP move GridFTP exit FTP- HRM Layer Shared memory Corba 1a1b 2a 2b 3a 3b 1a: stor/retv 1b: hrm_get/hrm_put 2b: call_back 2a: unblock semaphore 3a: success_code 3b: hrm_release

Computing Sciences Directorate, L B N L 19 CHEP 2003 Types of Spaces and Files Space reservation servicesSpace reservation services Spaces and files: volatile, durable, permanent Lifetime, action at end of lifetime Volatile – SRM owned, files can be removed if space needed Durable – files cannot be removed, but administrator notified Permanent – can be removed by owner only Directory servicesDirectory services Usual unix semantics any type of files in directory Access control servicesAccess control services Support owner/group/world permission Can only be assigned by owner File sharing for read-only files check with source for shared file permission File sharing for updatable files check with “master copy” for time of last update

Computing Sciences Directorate, L B N L 20 CHEP 2003 File movement functionality: srmGet, srmPut, srmReplicate SRMClient Client-FTP-get (pull) Client-FTP-put (push) srmGet/srmPut SRM-FTP-put (push) SRMClient SRM/ No-SRM SRM-FTP-get (pull) srmReplicate SRM/ No-SRM FTP-get

Computing Sciences Directorate, L B N L 21 CHEP 2003 SRM Methods File Movement srm(Prepare)Get: srm(Prepare)Put: srmReplicate: Lifetime management srmReleaseFiles: srmPutDone: srmExtendFileLifeTime: Terminate/resume srmAbortRequest: srmAbortFile srmSuspendRequest: srmResumeRequest: Space management srmReserveSpace srmReleaseSpace srmUpdateSpace srmCompactSpace: srmGetCurrentSpace: FileType management srmChangeFileType: Status/metadata srmGetRequestStatus: srmGetFileStatus: srmGetRequestSummary: srmGetRequestID: srmGetFilesMetaData: srmGetSpaceMetaData:

Computing Sciences Directorate, L B N L 22 CHEP 2003 Advantages of using SRMs Synchronization between storage resourcesSynchronization between storage resources Pinning file, releasing files Allocating space dynamically on as “needed basis” Insulate clients from storage and network system failuresInsulate clients from storage and network system failures Transient MSS failure Network failures Interruption of large file transfers Facilitate file sharingFacilitate file sharing Eliminate unnecessary file transfers Support “streaming model”Support “streaming model” Use space allocation policies by SRMs: no reservations needed Use explicit release by client for reuse of space Control number of concurrent file transfersControl number of concurrent file transfers From/to MSS – avoid flooding MSS and thrashing From/to network – avoid flooding and packet loss

Computing Sciences Directorate, L B N L 23 CHEP 2003 Ongoing and Future Work Ongoing workOngoing work Developing Standard SRM interfaces Particle Physics Data Grid (PPDG) project LBNL, TJNAF, FNAL European Data Grid (EDG) project WP2 - data management WP5 – mass storage (CASTOR) Deployment LBNL, BNL, ORNL, TJNAF, FNAL, CERN, (SE-England) Use of SRM by other agents Storage Resource Broker (SDSC) calling HRM to Stage files from HPSS GridFTP invoking HRM Future workFuture work Access authorization – community access service (CAS) “On-demand” space allocation, accounting, and charging Replica management – invoke SRMs and RLS as a single service Request executer (e.g. DAGMAN) to invoke SRMs SRMs over NeST (Network STorage)