Data Management The GSM-WG Perspective

Background SRM is the Storage Resource Manager A Control protocol for Mass Storage Systems Standard protocol: OGF GSM-WG –Chairs: A Shoshani (LBNL), J Jensen (STFC RAL)

Who Is Using It? EGEE, OSG,… WLCG … Not only academic HPC TIER 0 Country or ROC

Storage Element Implementations dCache (DESY+FNAL) DPM (CERN) CASTOR (CERN+RAL) BestMan (LBNL) Jasmine (JLAB) StoRM (INFN/ICTP-EGRID)

What is a “Storage Element”? Storage Element Disk storage Tape storage THE (gLite) GRID Computing Data Mgmt Metadata Disk storage Disk Pools

What is a “Storage Element” Interface to mass storage systems Part of a loosely coupled grid Provide uniform (control) interface Protect the storage system against the grid Enable grid to efficiently use storage

What is a Storage Element? Data Transfer Control Interface (SRM)‏ Information System Storage Element Used by EGEE, OSG,… Six-seven different implementations An SRM is a “Storage Element” but not vice versa GIN: Use SRB as Storage Element

Tech ~6 interoperating implementations –362 storage elements deployed –226 are SRM1 implementations –6 SRM2 implementations (can’t be quite right) Some Open Source, some not Using SOAP over GSI sockets –Delegation needed for srmCopy –No WS-Anything

Where is the File GUID SURL LFN Physical File TURL Physical File Storage Elements Physical File Physical File

Information View (simplified) Storage Area Storage Element VOInfo Control Protocol Data Transfer Protocol Space? Resource Accounting Owner/ Access

How is the file? Capa- bilities Expiration Mode Release When expired Warn When expired Perma- nent Retention Policy ReplicaOutputCustodial Access Latency OnlineNearline (Offline) (Lost) (…)

How I reach the file? File Access Parameters Overwrite Mode YesNo When Different Access Pattern TransferProcessing Connection WANLAN

Transfer Protocols WAN protocols –Must always support GridFTP –Suggestion to support HTTP/HTTPS GSI issue though - see tech slide dCache supports http for reading Support LAN protocols –Secure vs insecure

LCG: GFAL LCG (will) decide to use GFAL – the “Grid File Access Library” It was decided to interface to EDG SE using SRM 1 interface For now using EDG RM POSIX interface SRM 1 client EDG 2.1 Storage Element Mass Storage EDG SRM

EGEE: DICOM server support The Grid Storage Element WP10 DM2 DICOM Server Metadata Encrypt, anonymise Metadata Store keyStore patient metadata Access control on metadata required; different ACLs for different types of metadata

Challenges Users State of the Art Middleware Economy, Sustainability Scale, Performance CapabilitiesServices

Users Usability Customer mgmt Culture, History Applications

Services DiscoveryAccountingAvailabilityTrust

State of the Art MediaVirtualisation Web Services

Middleware Ease of install And Config Maintenance Support ApplicationsStability

Scale, Performance Scale, Performance Volume Number of files Size of files Transfer rates Staging

Sustainability, Economy Economy Cost Model AgreementDynamicTrustScale

Capabilities SECURITYCurationAccessContent