J Jensen CCLRC RAL Data Management AUZN (mostly about SRM though) GGF 16, Athens J Jensen.

Slides:



Advertisements
Similar presentations
30-31 Jan 2003J G Jensen, RAL/WP5 Storage Elephant Grid Access to Mass Storage.
Advertisements

Jens G Jensen CCLRC e-Science Single Sign-on to the Grid Federated Access and Integrated Identity Management.
Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities.
CASTOR SRM v1.1 experience Presentation at HEPiX MSS Forum 28/05/2004 Olof Bärring, CERN-IT.
HEPiX GFAL and LCG data management Jean-Philippe Baud CERN/IT/GD.
Steve Traylen Particle Physics Department Experiences of DCache at RAL UK HEP Sysman, 11/11/04 Steve Traylen
Jens G Jensen CCLRC/RAL hepsysman 2005Storage Middleware SRM 2.1 issues hepsysman Oxford 5 Dec 2005.
Data Management Expert Panel. RLS Globus-EDG Replica Location Service u Joint Design in the form of the Giggle architecture u Reference Implementation.
EGEE is a project funded by the European Union under contract IST Using SRM: DPM and dCache G.Donvito,V.Spinoso INFN Bari
Storage: Futures Flavia Donno CERN/IT WLCG Grid Deployment Board, CERN 8 October 2008.
CASTOR SRM v1.1 experience Presentation at SRM meeting 01/09/2004, Berkeley Olof Bärring, CERN-IT.
Heads in the cloud? GSM-WG at OGF31, Taipei Jens Jensen, RAL.
Andrew McNab - SlashGrid, HTTPS, fileGridSite SlashGrid, HTTPS and fileGridSite 30 October 2002 Andrew McNab, University of Manchester
Maarten Litmaath (CERN), EGEE User Forum, CERN, 2006/03/02 (v3) Use of the SRM interface Use case What is the SRM? –Who develops it? –Is it a standard?
CERN IT Department CH-1211 Genève 23 Switzerland t Plans and Architectural Options for Physics Data Analysis at CERN D. Duellmann, A. Pace.
INFSO-RI Enabling Grids for E-sciencE gLite Data Management Services - Overview Mike Mineter National e-Science Centre, Edinburgh.
StoRM Some basics and a comparison with DPM Wahid Bhimji University of Edinburgh GridPP Storage Workshop 31-Mar-101Wahid Bhimji – StoRM.
Ákos FROHNER – DataGrid Security Requirements n° 1 Security Group D7.5 Document and Open Issues
Mass Storage System Forum HEPiX Vancouver, 24/10/2003 Don Petravick (FNAL) Olof Bärring (CERN)
Data Management The GSM-WG Perspective. Background SRM is the Storage Resource Manager A Control protocol for Mass Storage Systems Standard protocol:
D C a c h e Michael Ernst Patrick Fuhrmann Tigran Mkrtchyan d C a c h e M. Ernst, P. Fuhrmann, T. Mkrtchyan Chep 2003 Chep2003 UCSD, California.
Δ Storage Middleware GridPP10 What’s new since GridPP9? CERN, June 2004.
ASPiS Security Jens Jensen Science and Technology Facilities Council AHM, 8-11 Sep 2008 Edinburgh.
Author - Title- Date - n° 1 Partner Logo EU DataGrid, Work Package 5 The Storage Element.
Author - Title- Date - n° 1 Partner Logo WP5 Summary Paris John Gordon WP5 6th March 2002.
Enabling Grids for E-sciencE Introduction Data Management Jan Just Keijser Nikhef Grid Tutorial, November 2008.
Jens G Jensen e-Science Centre hepsysmanix HEPiX report for hepsysman RAL, 10 May 2006.
Light weight Disk Pool Manager experience and future plans Jean-Philippe Baud, IT-GD, CERN September 2005.
Owen SyngeTitle of TalkSlide 1 Storage Management Owen Synge – Developer, Packager, and first line support to System Administrators. Talks Scope –GridPP.
11/5/2001WP5 UKHEPGRID1 WP5 Mass Storage UK HEPGrid UCL 11th May Tim Folkes, RAL
INFSO-RI Enabling Grids for E-sciencE gLite Data Management and Interoperability Peter Kunszt (JRA1 DM Cluster) 2 nd EGEE Conference,
SRM & SE Jens G Jensen WP5 ATF, December Collaborators Rutherford Appleton (ATLAS datastore) CERN (CASTOR) Fermilab Jefferson Lab Lawrence Berkeley.
Jens G Jensen RAL, EDG WP5 Storage Element Overview DataGrid Project Conference Heidelberg, 26 Sep-01 Oct 2003.
Derek Ross E-Science Department DCache Deployment at Tier1A UK HEP Sysman April 2005.
INFSO-RI Enabling Grids for E-sciencE Introduction Data Management Ron Trompert SARA Grid Tutorial, September 2007.
Storage and Data Movement at FNAL D. Petravick CHEP 2003.
10 May 2001WP6 Testbed Meeting1 WP5 - Mass Storage Management Jean-Philippe Baud PDP/IT/CERN.
EGI-Engage Data Services and Solutions Part 1: Data in the Grid Vincenzo Spinoso EGI.eu/INFN Data Services.
Andrew McNab - Dynamic Accounts - 2 July 2002 Dynamic Accounts in TB1.3 What we could do with what we’ve got now... Andrew McNab, University of Manchester.
SRM-2 Road Map and CASTOR Certification Shaun de Witt 3/3/08.
Distributed Data Access Control Mechanisms and the SRM Peter Kunszt Manager Swiss Grid Initiative Swiss National Supercomputing Centre CSCS GGF Grid Data.
SESEC Storage Element (In)Security hepsysman, RAL 0-1 July 2009 Jens Jensen.
DMLite GridFTP frontend Andrey Kiryanov IT/SDC 13/12/2013.
Grid Deployment Board 5 December 2007 GSSD Status Report Flavia Donno CERN/IT-GD.
GridPP2 Data Management work area J Jensen / RAL GridPP2 Data Management Work Area – Part 2 Mass storage & local storage mgmt J Jensen
Storage Element Security Jens G Jensen, WP5 Barcelona, May 2003.
Martina Franca (TA), 07 November Installazione, configurazione, testing e troubleshooting di Storage Element.
Introduction to Storage Element Hsin-Wei Wu Academia Sinica Grid Computing Center, Taiwan.
Security recommendations DPM Jean-Philippe Baud CERN/IT.
9/20/04Storage Resource Manager, Timur Perelmutov, Jon Bakken, Don Petravick, Fermilab 1 Storage Resource Manager Timur Perelmutov Jon Bakken Don Petravick.
Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT Standard Protocols in DPM Ricardo Rocha.
User Domain Storage Elements SURL  TURL LFC Domain (LCG File Catalogue) SA1 – Data Grid Interoperation Enabling Grids for E-sciencE EGEE-III INFSO-RI
J Jensen / WP5 /RAL UCL 4/5 March 2004 GridPP / DataGrid wrap-up Mass Storage Management J Jensen
Security recommendations for dCache
EGEE Data Management Services
DPM Installation Configuration
Classic Storage Element
StoRM: a SRM solution for disk based storage systems
Vincenzo Spinoso EGI.eu/INFN
Status of the SRM 2.2 MoU extension
GGF OGSA-WG, Data Use Cases Peter Kunszt Middleware Activity, Data Management Cluster EGEE is a project funded by the European.
Data services on the NGS
SRM v2.2 / v3 meeting report SRM v2.2 meeting Aug. 29
Introduction to Data Management in EGI
SRM Developers' Response to Enhancement Requests
Introduction to reading and writing files in Grid
The INFN Tier-1 Storage Implementation
Stephen Burke, PPARC/RAL Jeff Templon, NIKHEF
Data Management cluster summary
INFNGRID Workshop – Bari, Italy, October 2004
Presentation transcript:

J Jensen CCLRC RAL Data Management AUZN (mostly about SRM though) GGF 16, Athens J Jensen

CCLRC RAL GIN! Gimme Interoperability Now!! SRB ISLANDSRM ISLAND

J Jensen CCLRC RAL SRB - IANASRBE SRB is not SRM –Different aims, different users –SRB provides its own Data Grid AUCN: –Username/password –GSI for S commands (if compiled in) –Define id mapping… Access control replicated with data –Group permissions

J Jensen CCLRC RAL SRM Overview SRM is a file control protocol –GGF standard – GSM-WG –SOAP/HTTP over GSI sockets Something else does the transfer –WAN: Usually GridFTP –LAN: local protocol (RFIO, DCAP,…)

J Jensen CCLRC RAL Implementations Special ones – for specific tape MSS –JLAB, LBNL, CERN/RAL General purpose (usually to disk) –DPM from CERN/LCG, –dCache from DESY/FNAL, –StoRM from INFN

J Jensen CCLRC RAL SRM Versions Designers: –ACL not a major priority Implementers: –Listen to users (often) Users: –ACL not a major priority (HEP) Version 1.1 –Secure (GSI), but… –No functions for ACL Version 2.1 –Unixy +rwxrwxrwx –…POSIX

J Jensen CCLRC RAL Implementations provides SRM SRM1.1SRM2.1 dCacheYESNot seen yet DPMYES CASTOR1YESNO CASTOR2NOYES

J Jensen CCLRC RAL File Transfer Area Implementation LANWAN dCacheDCAPGridFTP DPMRFIOGridFTP CASTORRFIOGridFTP

J Jensen CCLRC RAL Local Protocols Traditional insecure versions… –Use Unix UID for authentication –No data confidentiality (encryption) Both RFIO and DCAP have GSI versions –Not always used by default –Need hostcerts for pool nodes –Dont necessarily encrypt –GSI/SSL negotiations slow

J Jensen CCLRC RAL GridFTP Implementations Use GSI authentication Authorise by DN, using gridmap files Dont encrypt data by default –Or large transfers would be slow

J Jensen CCLRC RAL DPM 1.5 Improved Security Integrated access control in nameserver –GridFTP, SRM, RFIO: consistent ACL RFIO –GSI only –No Encryption Performance vs confidentiality POSIX ACLs VOMS

J Jensen CCLRC RAL StoRM Security Requires ACL capable filesystem –GPFS (, ext3, ReiserFS,…) Being tested by INFN CNAF

J Jensen CCLRC RAL CASTOR 2 SRM Access control not implemented yet Will rely on CASTOR for ACL

J Jensen CCLRC RAL SRM Data Movers: Gaps Data movers must update ACLs when moving data –Support SRM 2.1 –Some copy as user (delegated) –Some as a service Not quite trivial –Data movers dont have special privileges

J Jensen CCLRC RAL Back Doors? File written via Grid can sometimes be read with local protocol –Or via SRM 1.1? Privileged (root/admin) access –Storage Filename is often random –Rarely a concern

J Jensen CCLRC RAL Conclusions GIN: Two Islands – SRM and SRB WAN protocols secure (sort of) –But no data encryption by default Increasingly, LAN protocols are secured Implementations are available (sort of) –SRM 1.1 is still widely used